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@ Neural computer. 



© An information processing system includes a plurality of functional blocks (neurons) (100) and a data bus 
GOO) for transmitting in common the outputs of the individual functional blocks (neurons). Data transaction 
^ among the functional blocks (neurons) is performed through the data bus on the tin^Ms.on baas For 
S preventing the outputs from conflicting or competition, addresses are assigned to the ind.vjdua^locks (neurons) 
respectively, so that only the functional block (neuron) having the own address designated by the address signal 
^supplied through an address bus (302) outputs data signal onto the data bus. while the other funct.onal block s 
^(neurons) receive the information on the data bus as the signal originating in the functional block whose address 
„Js designated at that time point. The addresses are sequentially changed. During a round p the addresses 
{5 signals, data are transmitted from given functional blocks (neurons) to other given functional blocks (neurqns). 
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INFORMATION PROCESSING SYSTEM 

BACKGROUND OF THE INVENTION- 

The present invention generally relates to an information processing system or apparatus such as an ■ 
electronic computer or the like. More particularly, the invention is concerned with an information processing 

5 system or apparatus and a neural computer such as an analogue computer, neural network and the like 
which are capable of performing information processings with a high degree of parallel operation. 

A so-called analogue computer includes a plurality of functional blocks such as address, integrators and 
others, wherein each of the functional blocks receives signals outputted from the otirer functional block or 
blocks directly or by way of a scaler circuit having a multiplication function, a sign inverter and others. The 

to functional block then performs arithmetic operations such as addition, integration and other on the signal as 
received to thereby determine its own output value which is then transmitted to the other functional block or 
blocks. Usually, it is determined by operator in accordance with problems as imposed which of the 
functional block's output is to be inputted to which of the functional blocks and what a numerical value is to 
be set in the scaler circuit. 

15 In the network known as a neural net or network, there are provided, as shown in Fig, 1 of the 
accompanying drawings/a plurality of cell bodies and functional circuits 10 referred to as dendrites, wherein 
each of the functional circuits receives signals outputted from the other functional circuit through the 
medium of a scaler 11. Each of the functional blocks each composed of the functional circuit 10 and the 
scaler circuit 1 1 arithmetically determines a total sum of the values represented by the signals as its own 

20 internal value and outputs a signal corresponding to the internal value to the other functional blocks. 
Selection of the coefficients (hereinafter referred to as the weight values) for the individual scalers are 
automatically determined through a process usually referred to as the learning. An example of this type 
neural network is disclosed in a Japanese periodical "NIKKEI ELECTRONICS", No. 427 (August 10. 1987), 
pp. 115-124. 

25 An attempt for implementing the neural network in the form of a semiconductor integerated circuit is. 
disclosed in a Japanese periodical "NIKKEI MICRODEVICE", July 1988, pp. 44-89. One of such semicon- 
ductor integrated circuits for the neural network is also discussed in "A CMOS ASSOCIATIVE MEMORY 
CHIP BASED ON NEURAL NETWORKS": 1987 ISSCC DIGEST OF TECHNICAL PAPERS, pp. 304-305. In 
"NIKKEI MICRODEVICE", July, 1988, pp. 72 to 78, there is proposed a neural network by using digital 

30 circuits. The digital circuits are easy to be integrated and can ensure a high accuracy for operations by 
virtue of high insusceptibility to noise and manufacturing tolerance. Disadvantageous -however is the digital 
circuit in that a greater number of parts or elements are required when compared with the analogue circuit. 
Under the circumstance, in the case of the abovementioned approach, the numerical values which are 
usually expressed by pulse codes are expressed in terms of pulse numbers for simplification of the circuit 

35 configuration. 



SUMMARY OF THE INVENTION 

40 When complicated information processing is to be executed by using the analogue computer and the 
neural network, it becomes necessary to increase the number of the functional blocks such as adders, 
integrators and others, which in turn means that the amount of wiring conductors for interconnecting the 
functional blocks and the number of the scalers and others have to be correspondingly increased. 
Representing by N the number of the functional blocks, the number of the circuit points for interconnection 

45 of the functional blocks amounts to N x (n - 1). In other words, there is required an amount of hardware for 
the wiring conductors, scalers, etc. which is approximately in proportion to the second power of the number 
of the functional blocks. Accordingly, when the number of the functional blocks exceeds a certain value, the 
amount of hardware increases intolerably, making it practically impossible to implement the system. In 
particular, when the system is to be integrated by using the digital circuits which ensure high operation 

so accuracy and noise resistance and are suited for the integration, the abovementioned problem becomes 
more serious because a greater number of elements are required for constituting the scalers and others. 
Besides, an approach for expressing the numerical values in the pulse numbers in an effort to reduce the 
amount of hardware involves intolerably a lot of time for the processing. By way of example, even the 
numerical value of third order of magnitude requires for the expression thereof about one thousand times as 
long as the time required for the expression by the pulse code. 
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A first object of the present invention is to provide an information processing system or apparatus for 
a neural network and an analogue computer in which the amount of hardware is prevented from increasing 
rapidly notwithstanding of increase in the number of the functional blocks. 

A second object of the present invention is to provide a large scale neural network implemented by 
using digital circuits. 

A third object of the present invention is to provide an information processing system or apparatus 
capable of structuring a large scale neural network without lowering the yield "rate regardless of increasing 
the number of neurons. 

A fourth object of the present invention is to provide an information processing system or apparatus 
capable of implementing a high-speed large scale neural network while suppressing degradation in the 
operating speed due to increasing in the number of neurons. 

A fifth object of the present invention is to provide an information processing system or apparatus 
capable of realizing a large scale neural network by using a relatively small number of different photomasks. 

A sixth object of the present invention is to provide a neural network for an analogue computer for 
digital communication in which the weight values as well as the output values of the neuron circuits can be 
expressed in a multiplicity of bits and which allows data processings of large scale without need for 
increasing the area for implementing the neural network. 

A seventh object of the present invention is to provide an information processing system or apparatus 
which allows a learning process to be executed normally even when fault should take place in a learning 
circuit system and which allows a neural network to be realized with an excellently high integration density. 

In view of the above and other objects which will be apparent as description proceeds, there is provided 
according to a first general aspect of the present invention an information processing system (apparatus) 
which includes a plurality of functional blocks (neurons) and a bus for transmitting in common the outputs of 
the individual functional blocks (neurons), wherein data transaction among the functional blocks (neurons) is 
performed through the data bus on a time-division basis. For preventing the outputs from conflicting or 
competition, addresses are assigned to the individual blocks (neurons), respectively, so that only the 
functional block (neuron) having the own address designated by the address signal supplied through an 
address bus can output data signal onto the data bus. while the other functional blocks (neurons) receives 
the information on the data bus as the signal originating in the functional block whose address is designated 
at that time point. The addresses are sequentially changed. In this way. in the course of a round of the 
addresses signal, data can be transmitted from given functional blocks (neurons) to other given functional 
blocks (neurons). With this arrangement, the functional block (neuron) for receiving the signals can use the 
scaler (synapse) and others in common to a large number of counterpart or member blocks, whereby the 
amount of hardware for the circuit become essentially in proportion to the number of the functional blocks, 
as the result of which increasing in the number of the functional blocks (neurons) is no more attended with 
rapid or abrupt increase in the amount of hardware. 

It is taught according to another aspect of the invention to provide a semiconductor memory for stonng 
the weight values only in a predetermined number sequentially in the order starting from the maximum 
absolute value of the weight. By storing the weight values in the semiconductor memory, exchange of the 
weight values can be facilitated. Further, by storing only a predetermined number of the weight values 
starting from the greatest value, the amount of hardware can be correspondingly reduced. Besides, by 
storing the weight values in the order in which they are accessed, it is possible to read out previously the 
weight value next to be accessed, whereby the operation speed can be enhanced. 

According to a second general aspect featuring the present invention, the bus for mutually connecting 
the functional blocks is implemented in a hierarchical structure. By virtue of this feature, it is possible to 
prevent the load applied to one bus from becoming heavier even when the number of neurons is increased, 
whereby a large scale neural network can be realized. In this conjunction, it is noted that when the neural 
network is implemented in the form of a WSI (wafer scale integrated circuit), the hierarchical structurization 
of the buses is attended with a problem that occurrence of faults such as breakage, short-circuits and the 
like in the bus of higher hierarchical level exerts adverse influence over a wide range and lowers the yield 
rate This problem can be solved by increasing the width of the wiring conductors for the bus of higher 
hierarchical level as well as the space between the bus conductors to thereby reduce the fault rat.o. 

By pipelining the signal transmission over the bus. it becomes possible to send signals from several 
neurons to the other neurons within a time within which a signal is sent from one neuron to the other 
neurons, whereby the operation speed can be increased. ... 

In a preferred embodiment of the present invention, parts of the buses are sifted in posit.on for 
connection to adjacent blocks. With this arrangement, the integrated circuits of the utterly same conf.gura- 
tion disposed in an array can be supplied with signals different from one to another integrated circuit. In 
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other words, it is possible to realize a WSI (wafer scale integrated circuit) by arraying the functional blocks 
(neurons) formed by using a same mask. 

In an improved embodiment of the pr sent invention, a learning circuit is provided separately from and 
in common to the individual neurons (functional blocks), and realized in a structure in which fault can be 

removed. u . . , 

In another preferred embodiment of the invention, the inter-neuron data transmission is performed by, 
using analogue signal, wherein determination of products and a sum thereof (multiplication and addition 
operation) is carried out through analogue calculation or alternatively the multiplication is carried out by an 
analogue calculation with the addition being performed by digital calculation. With this arrangement, the 
neuron circuit can be realized with a smaller number of constituent elements. Because of the inter-neuron 
data transfer by the digital signal, there arises no problem that the signal undergoes attenuation due to the 
resistance of the wiring conductors, as in the case of the data transfer by the analogue signal, even when a 
great distance intervenes between the individual neurons (functional blocks) in a large scale neural network. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a schematic circuit diagram showing a structure of the prior art information processing 
system; 

20 Fig. 2 is a schematic block diagram showing a general arrangement of an information processing 

system (apparatus) according to an exemplary embodiment of the present invention; 

Fig. 3 is a block diagram showing in detail a structure of one functional block (100) which is 
constituted by a functional circuit, a scaler circuit and a time division control circuit; 

Figs. 4A and 4B are diagrams showing circuit configurations of address decoders, respectively, which 
25 can be employed in the system shown in Fig. 2; 

Fig. 5 is a timing diagram for illustrating operation of an exemplary embodiment of the invention; 

Figs. 6A and 6B are diagrams showing, respectively, examples of a function transformation circuit 
required when the information processing apparatus according to the invention is to be used as a neural 
network; 

30 Fig. 7 is a diagram showing a neural network implemented by applying the information processing 

system according to an embodiment of the intention; 

Fig. 8 is a timing chart for illustrating operation of the neural network shown in Fig. 7; 
Fig 9 is a schematic circuit diagram showing an improved portion of the information processing 
system shown in Fig. 2 or 7 according to another embodiment of the invention in which buses are 
35 implemented in a hierarchical structure; 

Fig! 10 is a schematic circuit for illustrating a method of interconnecting the buses of hierarchical 
structure according to an embodiment of the invention; 

Fig. 11 shows a neuron network of an improved structure according to another embodiment of the 
invention which is arranged to decrease the processing time; 
40 Fig. 12 is a schematic diagram showing an improved structure of the neural network of hierarchical 

bus structure according to still another embodiment of the invention; 

Fig. 13 is a schematic view showing the information processing system according to the invention 
which is realized in the form of a WSI (wafer scale integration circuit); 
Fig. 14 is a timing chart for illustrating operation of the same; 
45 Fig. 15 is a circuit diagram showing in detail a portion of the embodiment shown in Fig. 13; 

Fig. 16 is a view showing a structure of the information processing system integrated in the form of a 
WSI according to yet another embodiment of the invention; 

Fig. 17 shows an address signal translation block of the system shown in Fig. 16; 
Figs. 18A and 18B show functional blocks in the system shown in Fig. 16; 
50 Fig. 19 is a view for illustrating operation of the system shown in Fig. 16; 

Fig 20 shows another example of the structure of a functional block; 

Fig. 21 shows a structure of a weight value memory circuit (101) incorporated in the functional block 
or neuron (100) shown in Figs. 3 and 7; 

Fig. 22 is a block diagram showing in further detail a structure of the functional block or neuron (100) 

55 shown in Figs. 3 and 7; m 

Fig. 23 shows in a schematic circuit diagram an improved structure of the information processing 
system of a multiplexed bus structure according to a further embodiment of the invention; 

Fig. 24A is a schematic circuit diagram showing an exemplary structure of the functional block or 
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neuron suited for the multiplexed bus structure according to a further embodiment of the invention; 

Fig. 24B is a schematic circuit diagram of an address decoder shown in Fig. 24A; 

Fig. 25 is a schematic diagram for illustrating learning algorithm based on a back propagation 
method; 

s Fig. 26 is a schematic diagram for illustrating the concept underlying a separately provided learning 

circuit of the information processing system according to another embodiment of the invention; 

Fig. 27 is a diagram for illustrating in more detail an exemplary structure of the learning circuit 
system shown in Fig. 26; 

Fig. 28 is a circuit diagram for illustrating details of the neuron (100) shown in Fig. 27; 
to Fig. 29 is a circuit diagram for illustrating details of the learning circuit system (999) shown in Fig. 27; 

Fig. 30 is a diagram showing an exemplary structure for remedying a fault of the learning circuit; 
I Figs. 31 to 34 are schematic diagrams showing other improved exemplary embodiments for 

remedying fault, respectively; 

Fig. 35 is a schematic diagram showing yet another embodiment of the invention in which a plurality 

;s of wafers are stacked; 

Fig. 36A is a schematic diagram showing an exemplary embodiment of the neural network in which 
inter-neuron data transmission is performed by making use of the digital signal while intra-neuron product 
summing calculation is performed by the using analogue signals; 

Fig. 36B is a circuit diagram showing a modification of the embodiment shown in Fig. 36A; 
20 Fig. 37A is a view for illustrating the concept of a non-linear translation or conversion circuit; 

Fig. 37B shows a memory map exemplifying a structure of the same; 

Fig. 38 is a schematic circuit diagram showing, by way of example, a circuit configuration of a digital 
PWM converter; 

Fig. 39 is a schematic circuit diagram showing a neural network according to another exemplary 
25 embodiment of the invention; 

Fig. 40A is a circuit diagram showing a configuration of a half precharge circuit; 
Fig. 40B is a timing chart for illustrating operation of the half precharge circuit; and 
Figs. 41. 42, 43, 44, 45 and 46 are views illustrating other exemplary embodiments of the present 
invention, respectively. 
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DESCRIPTION OF THE PREFERRED EMBODIMENTS 



Fig. 2 is a schematic block diagram showing a general arrangement of an information processing 
system according to an exemplary embodiment of the present invention. In this figure, a reference numeral 

35 10 denotes a functional unit, a numeral 11 denotes a scaler circuit, a numeral 12 denotes a time division 
control circuit, a numeral 300 denotes a data input/output bus, a numeral 302 denotes an address bus, a 
numeral 303 denotes a clock bus, and a reference numeral 120 denotes a timing generation circuit. Fig. 3 is 
a block diagram showing in further detail the structure of one functional block (also referred to as a neuron) 
100 which is constituted by the functional circuit 10. the scalar circuit 1 1 and the time division control circuit 

40 12. As can be seen in Fig. 3. the functional circuit 10 is composed of an adder 103, latch circuit 104 and 
* 105 and a buffer circuit 106 having tri-state outputs. The scaler circuit 11 is constituted by a multiplier 102. 
The time-division control circuit 12 is composed of address decorders 107 and 107 and a memory circuit 
101. 

The address decoder 107' serves to detect that the address signal sent via the address bus 302 has 
45 made a round, while the address decoder 107 serves to make decision as to whether or not the address 
signal is destined to the functional block under consideration. More specifically, assuming that the number 

.«* of the functional block is represented by N. then the addresses "1 "2", "3". .... "N\ by way of example, 
are assigned to 100 blocks each constituted by the functional circuit 10. the scaler 11 and the time^ division 
circuit 12. wherein the address signals are repeated in the sequence of "0", rt 1 rt , "2", "3" "N". "0". "1", 

- so "2", "3" and so forth, wherein the address n Q" is assigned to none of the functional blocks and made use of 
for detecting that the address signals enumerated above has made one round. When the address signal is 
"0", there is generated an output signal which indicates that the address decoders 107 incorporated in all 
the time division control circuits 12 are selected. When the address signal assumes one of the other values 
than "0", a signal is outputted which indicates that only the address decoder 107 incorporated in the time 
55 division control circuit assigned with that one address is selected, whereby the buffer circuit 106 paired with 
the address decoder 107 and incorporated in the functional circuit is activated to output a signal onto the 
data bus. The outputs of the buffer circuits 106 of the other functional blocks remain in the state of high 
resistance. 
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The memory circuit 101 serves to store therein coefficients (weight values) for coupling between the 
functional block paired with the memory circuit 101 and other functional blocks and outputs the coupling 
coefficient (weight value) when the output of a functional block designated by the address s,gna. at a given 
time point inputted to the functional block paired with the memory circuit. At that time pomt. the output 

s uZ Si functional block designated by the address signal is present on the data bus 300. Accordingly the 
mu«ip5er 102 outputs a product of the signal from the abovementioned functional block and the coupling .. 
cTfEent (weight value) whereon the product signal is applied to the adder 103. When the clock signal is 
appnS at this time point via the clock bus 303. the results of the addition accumulated ,n the latch circuit 
104 up to that time point is added with the output of the multiplier 102. the sum thus obtained I being then 

,o stored in the latch circuit 104 as the updated accumulated value. Consequently, when the address signal 
has turned a round, the sum of the outputs from all the functional blocks weighted w,th the coup ng 
coefficients, respectively, is obtained at the output of the latch circuit 104. The output of the latch arcurt : 104 
at that time point is then stored in the latch circuit 105 to be retained therein as an internal value of the 
associated functional block during a period until the address signal turns a round subsequent y. Parenthet.- 

,b cany it is to be mentioned that by resetting the content of the latch circuit 104 every times the addresses 
have turned a round, the functional block under consideration operates, so to say as an adder of an 
analogue computer, while it operates as an integrator by accumulating the contents of the latch , c.rcu,t 104 
instead of resetting. Fig. 4A and 4B show, by way of example only, circuit configurations 
decoders, respectively. More specifically, the circuit shown in F,g. 4A is used as the address decoder 107 

20 mentioned above, the output of which assumes a high level only when all the inputs are 0 i.e. when the 
address signal indicates the address "0". i.e. when the address signal indicates the address 0 and 
otherwise assumes a low level. On the other hand, the circuit shown in Fig. 4B .s employed as the , address 
decoder 107 the output of which assumes the high level only when the address signal indicates its own 
address (i.e. the address allocated to that decoder 107) and otherwise assumes the low level The. address 

25 of concern can be established in an arbitrary manner by placing an inverted b,t(s) at a g.ven b.t pos,t.on(s) 

° f TnThfca S se i9 o n f a, the information processing system according to the illustrated embodiment of the 
invention the number (N - 1) of the scaler circuits which have heretofore been required to be mcorporated 
in each of the functional blocks in a number corresponding to that of the other functional bloc s ,n he prior 

ao art system can be reduced to one for each of the functional blocks. Besides, according to . the .nustrat 
embodiment, the number of the address which have heretofore been requ,red to be ^corporate n each of 
the functional blocks in a number^ - 1) corresponding to that of the other functional blocks can be 
decreased down to one for each of the functional blocks. As a result, a numencal value .^pressed I 
number of pu.ses can be expressed in the form of a pulse code. This in turn means that 

35 processing time which has heretofore given rise to a problem in the time d.v.s,on processmg ,s canceled 

OUt lTde a So 9 n e wi.. be made in concrete of an application of the system elucidated above an 
analogue computer. In this conjunction, let's suppose, by way of example, that the followng s.mu.taneous 
differential equations are to be solved. 
40 gtX = aX + bY (1) 

In *is tase thL arfemployed two functional blocks 100 each composed of the functional circuit 10. the 
scater Srcu it 11 and the time division circuit 12. wherein one of the functiona. blocks 1" « a»gn»d «rth 
tteaddVess »1» with the other allocated with the address "2". Since both the functional blocks WO are 
45 ooerated as the integrators, respectively, the latch circuit 104 incorporated in these functional blocks 1 00a 
' not reset Tn the couTse of ca.cuW.on. Further, the output of the functiona. b.ock assigned with ^ address 
»?•• s defined to be X while that of the functiona. b.ock having the address "2" is defined to be Y On these 
conditions. L inputs to the functiona. blocks or the integrators are represented by the d.fferentia.s of the 
outputs, respectively. Accordingly, the following expressions apply valid: 

50 Accordingly arrangement may be made such that the signal applied to the functional circuit 10 through the 
staler S t 11 can be given by the right-hand side of the expression (1) or (2) mentioned above. More 
soecTficT when the pace or rate (indent) of the time for the integrating calculation ,s represented by r. the 
101 assigned with the address "1" is placed with "0" at the ajo* .address. 

55 address and rb at the second address, while the memory circurt assigned with the address 2 is placed 
w^thTrthe zero-th address, rc at the first address and rd at the second address. Subsequently he 
Men circuits 104 are placed with the initial values of X and Y. respectively. Now. the preparation for *e 
ntfgrS has been completed. Operation for the calculation is effectuated by generating by means of the 
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timing generation circuit 102 the address signals repeating "0", T, "2", "0", "V\ "2 rt . "0". "1 M and so 
forth every frame onto the address bus 302 and the clock signal containing one pulse for each frame onto 
the clock bus 303. On the conditions, the address signal is "0 n in the first frame. Accordingly, the outputs of 
both the memory circuits 101 one of which is shown in Fig. 3 are "0". Consequently, the outputs of both the 
5 multipliers 102 are also "0". The output of the adder 103 is equal to the output of the associated latch 
circuit 104, i.e. the initial value of X or Y. At that time, application of the clock signal to the latch circuit 104 
does not alter the state of the latch circuit 104 in which the initial value of X or Y is held, i.e. the initial value 
of X or Y remains unchanged. On the other hand, the latch circuit 105 fetches the output of the latch circuit 
104. i.e. the initial value of X or Y in response to the clock signal contained in the instant frame, 
v w Subsequently, in the second frame, the address signal becomes "1". resulting in that the buffer circuit 106 
of the functional block assigned with the block address of T is activated, while the content of the latch 
v circuit 105 of the block address "I", i.e. the initial value of X is outputted onto the data bus. Further, since 
the address signal is "1", the memory circuit 101 of the block address T outputs w ra", and the memory 
circuit 101 of the block address w 2 n outputs "tc". Consequently, the outputs of the multipliers 102 are 
75 "raX w and "rcX", respectively. Accordingly, the outputs of the address 103 are "X + raX" and "Y + tcX", 
respectively, which are then fetched by the latch circuits 104, respectively, in response to the clock signal 
r within the instant frame. Since no clock signal is applied to the latch circuits 105, they continue to remain in 
the state where the original values of X and Y are held as they are. Next, in the third frame, the address 
T signal becomes "2", and the initial value of Y is outputted onto the data bus, as the result of which the 
^20 memory circuits 101 output "rb" and Vd". respectively. Consequently, the adder 301 adds "rbY" or "rdY" 
to the output of the associated latch circuit 104 to thereby output "X + r (aX + bY) " or "Y + r (cX + dY)- 
As can be understood from the expressions (1) and (2), the above value represents the value of X or Y 
after lapse of the minute time r from the initial state. Within the instant frame (third frame), these values are 
fetched by the latch circuits 104 to be subsequently fetched by the latch circuits 105 as updated values of 
25 X and Y, respectively, within the fourth frame. Thereafter, the similar calculation is repeated. In this way, the 
simultaneous differential equations (1) and (2) can be solved at a pace of the time t. By diminishing the 
temporal pace or duration t, the time involved in the calculation is naturally increased. However, the 
accuracy of calculation can be improved correspondingly. Further, when a need arises for the adder in the 
sense used in conjunction with the analogue computer, a new value is written in the latch circuit 105 with 
30 the value of the latch circuit 104 being reset to zero within the frame for which the address signal is "0". 
Then, the functional block 10 can be operated as the adder. Thus, it will be appreciated that one and the 
same circuit can be employed as the integrator or alternatively as the adder by controlling correspondingly 
the generation of the reset signal mentioned above. Besides, it goes without saying that the multiplier 102 
and the adder 103 can be so operated as to be capable of handling the negative numbers in terms of 
35 corresponding complementary numbers. It should further be added that the latch circuit 105 is not 
necessarily required when the time pace t for the integration is selected to be sufficiently small. 

The circuits themselves of the analogue computer described above are realized by the digital circuits. 
Accordingly, although the instability such as drift or phenomena inherent to the analogue circuits can be 
avoided, consideration must be paid to the instability ascribable to the principle of the analogue computer 
40 such as the positive feedback and others. In the analogue computers known heretofore, the circuits 
themselves are realized by analogue circuits. For this reason, the cost of the analogue computer increases 
abruptly when the accuracy on the third or higher order of magnitude is to be realized. In reality., it has 
been unrealistic or impractical to realize the accuracy on the third or higher order of magnitude. However, in 
the case of the illustrated embodiment of the invention, the accuracy can readily be enhanced merely by 
45 increasing the number of the bits, since the circuits themselves are digital circuits. Besides, integration of 
the circuits is facilitated because of the digital nature. Heretofore, wiring of the functional blocks to one 
another has been carried out manually by using patch boxes or the like because of difficulty encountered in 
the integration of connector parts. Under the circumstance, the number of the functional blocks capable of 
^ being interconnected can not exceed one hundred at the most. In contrast, according to the instant 
r so embodiment of the invention, the wiring of the functioanl blocks to one another can be realized merely by 
writing the wiring information in the memory circuits 101 which can be easily integrated. Besides, the wiring 
information can be saved in a storage medium such as a magnetic disc or the like. Of course, the wiring 
information can also be automatically generated to be automatically written in the memories or storage 
equipment. It should further be added that in the case of the computer according to the instant 
55 embodiment, a plurality of the functional blocks operate simultaneously, which is advantageous in respect to 
the time involved in the calculation for solving simultaneous equations containing a large number of 
variables. 

Next, description will be directed to the use of the system as a so-called neutral net. In that case, the 
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functional block 100 shown in Fig. 3 corresponds to a single unit referred to as a neuron, so to say, while 
the scaler 1 1 corresponds to a synapse for receiving signals from the other neurons with the combination of 
the adder 103 and the latch circuit 104 corresponding to a so-called dendron (of the cell body). Weighting 
of the individual synapses is realized by storing the weight coefficients in the memory circuits 101. The 

5 negative weight coefficient provides a suppressing effect, while the positive weight coefficient provides an 
exciting effect. As the absolute value of the coefficient increases, the weight becomes more significant. 
Thus, by rewriting the contents of the memories 101, operation referred to as the learning is rendered 
possible, which will be described later on. In the case of the analogue computer, the numerical value 
represented by the output of the latch circuit 104 is utilized as the output value of the associated function 

70 block as it is (or by discarding the less significant bits). However, in the case of the neural net, the output 
value of the neuron 100 is given as the result of numerical transformation of the output of the latch circuit in 
accordance with the sigmoid function expressed by 
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F < x > - - HoT (3) 

1 + e 

or in accordance with a function approximating the sigmoid function by polygonal lines as follows: 



25 F(X) = 



1 : a < X 

X 1 



2a + 2 



: - a < X < a (4) 



0 : X <_ - a 



30 The circuit for performing the numerical transformation is provided between the latch circuits 104 and 105 
or between the latch circuit 105 and the buffer circuit 106. Alternatively, the numerical translation can be 
realized by a control apparatus (not shown) for controlling the whole system. The circuit for the numerical 
transformation in accordance with the polygonal function given by the expression (4) can be implemented in 
such a circuit configuration as shown in Fig. 6A, by way of example. In this figure, a reference numeral 610 

35 denotes input terminal and 61 1 denotes output terminal. Connected to one inputs of comparators 601 and 

602, a multiplier 603, selectors 604 and 605 and an adder 603 are the signals representative of numerical 
values "a", "-a", "1/2a", "I", "0" and "1/2", respectively. When the numerical value represented by the 
input signal applied to the input terminal 610 is greater than "a", the comparator 601 controls the selector 
604 so that the signal representative of "1" is outputted from the output terminal 611. When the numerical 
value represented by the input signal is not greater than "-a", the comparator 602 controls the selector 605 
so that a signal representative of the numerical value "0" is outputted through the output terminal 611. 
When the numerical value represented by the input signal lies between "a" and "-a", the signal passed 
through the multiplier 603 and the adder 606 is outputted to the output terminal 611. In this way, the 
numerical transformation in accordance with the function given by the expression (4) can be realized by 
using the circuit shown in Fig. 6A. The numerical transformation in accordance with the sigmoid function 
given by the expression (3) or other complicated functions can easily be realized by using a memory circuit 

603, as shown schematically in Fig. 6B. In this case, output values are previously calculated for the input 
values, respectively, and stored in the memory circuit 608 at addresses designated by the input signals, 
wherein the data stored in the memory circuit 608 are read out as the output values 611 to be utilized by 

so addressing the memory with the input signals 610. With this arrangement, the numerical transformation can 
be realized no matter how complex the functions to be resorted to are. 

Now, description will be made in detail of a neural network implemented according to an embodiment of 
the present invention by reference to Fig. 7. In this figure, the neuron is generally denoted by a reference 
numeral 100. For simplification of the description, only four neurons connected to one another are shown in 

55 Rg. 7. As described hereinbefore in conjunction with Fig. 3. each of the neurons includes a memory 101 for 
storing the weight value, a multiplier 102 for multiplying an input signal thereto with the weight valu , an 
adder 103 and a latch circuit 104 for determining a total sum of the products produced by the multiplier 
102. a latch circuit 105 for storing the abovementioned sum, a buffer circuit 106 having tri-state output, and 
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a decoder 107 for deciding whether or not the associated neuron 100 is selected. Each of the neurons 100 
is connected to a control apparatus 200 via a bus. In the case of the instant embodiment, the timing 
generator 120 shown in Fig. 2 and the function transformation circuit shown in Fig. 6A or 6B are 
incorporated in the control apparatus or controller 200. Further, reference numerals 301, 302. 303 and 304 
s denote buses for sending the input signal, address signal, clock signal and the cycle signal to each of the 
neuron 100 from the controller 200. A numeral 305 denotes a bus for sending the output signal from each of 
the neurons 100 to the controller 200. Although it is shown in Fig.. 7 that data bus 300 described 
hereinbefore is divided into the input bus 301 for the input signal and the output bus 305 for the output 

, signal it should be appreciated that the common data bus 300 described hereinbefore in conjunction with 

■■ ■10 Figs 2 and 3 may be used in place of them. Further, the cycle signal 304 may be generated on the basis 
of the clock signal 303 and the address signal 302 within each functional block (neuron) 100. as in the case 
-~ of the embodiments illustrated in Figs. 2 and 3. According to the instant embodiment, the latch circuit 104 is 
so implemented as to fetch the data in response to the edge of the clock signal applied through the clock 
bus 303 Each of the neurons 100 is assigned with an address inherent thereto (also referred to as the 
is neuron address or block address). Upon occurrence of coincidence between the neuron address and the 
address signal supplied from the control apparatus 200. the decoder 107 outputs a signal indicating the 
coincidence in the address. 

Next description will be directed to operation of the neuron network shown in Fig. 7 by reference to a 
timing chart illustrated in Fig. 8. Referring to Fig. 8. the controller 200 sends out the address signal which 
20 represents repeatedly a sequence of the addresses of the individual neurons, the clock signals for allowing 
the latch circuits 104 to fetch the data every time the relevant addresses are designated, and the cycle 
signal which causes the latch circuit 105 to fetch the data every time the sequence of the addresses has 
made a round, respectively. The address signal sent simultaneously to the individual neurons 100 from the 
control apparatus 200 via the address bus 302 are applied to the decoder circuits 107. whereon only the 
25 buffer circuit 106 of the neuron designated by the address signal outputs the signal onto the output bus 
305 The buffer circuits 106 of the other neurons remain in the state of high resistance (impedence). In th.s 
manner the content of the latch circuit 105 of the neuron 100 designated by the address signal is sent to 
the control apparatus 200. The control apparatus 200 sends out the signal of concern to each of the 
neurons 100 via the input bus 301 as it is or after transformation in accordance with the sigmoid function or 
30 an approximating polygon function. In each of the neurons 100. the weight value for the counterpart neuron 
designated by the address signal at that time point is read out from the memory 101. whereon the product 
of the weight value and the input signal supplied from the control apparatus 200 via the input bus 301 is 
determined by the multiplier 102. Subsequently, a sum of the product outputted from the multiplier 102 and 
the numerical value accumulated in the latch circuit 104 until then is determined by the adder 103. The sum 
35 signal thus produced is then applied to the latch circuit 104. The latter fetches the sum data in response to 
the application of the clock signal supplied via the clock bus 303. Thus, during a period in which the 
addresses make a round, the total sum of the signals from all the neurons and the we.ght value is 
determined in the latch circuit 104. Upon application of the cycle signal from the bus 304. the total sum is 
^ fetched by the latch circuit 105 and represents the updated output value of the associated neuron 100. In 
40 this way, the system shown in Fig. 7 operates as a neuron network. 

By the way. when equation containing many independent variables are to be solved or in case 
complicated information processing is to be performed, there are required a large number of the Junctional 
blocks (neurons). In such case, the connection of the neural network shown in Figs. 2 and I 7 is 
disadvantageous in that the data bus 300 (301. 305). the address bus 302 and the clock buses 303, 305 are 
4 5 imposed with excessive loads, making it difficult more or less to drive the system. Fig. 9 .s a schematic 
circuit diagram showing a portion of an improved information processing system according to another 
*- embodiment in which buses are implemented in a hierarchical structure to evade the problem mentioned 
above In this figure, one of functional block groups or neuron groups is shown. By interconnecting a large 
number of the circuits each of the configuration shown in Fig. 9 in such a manner as descnbed hereinafter 
a. so by reference to Figs. 10 to 13, there can be realized a large scale neural network. Now referring to Fig 9 
reference numerals 100 denote the neurons, respectively, and 301 to 305 denote the buses of first 
hierarchical level (the lowmost level). This portion may be implemented similarly to the neurons functional 
blocks) and the buses 301 to 305 shown in Figs. 3 and 7. However, the bit number of the address bus 302 
and the capacity of the memory 101 for storing the weight values have to be increased over those 
55 described hereinbefor in conjunction with Figs. 3 and 7 in order to be able to realize the large scale neural 
network. Further, in Fig. 9. reference numerals 311. 312. 313. 314 and 315 denote buses of a second 
hierarchical level for transmitting the input signal, address signal, clock signal, cycle signal and the output 
signals respectively. The buses 311 to 314 of those of the second hierarchical level send out the signals to 
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the buses 301 to 304 of the first hierarchical level, respectively, through a buffer circuit 153, while the 
output bus 315 of the second hierarchical level receives the signal from the output bus 305 of the first 
hierarchical level by way of a tri-state buffer circuit 156. A decoder 157 serves for deciding as to whether or 
not there exists the neuron (functional block) 100 designated by the address signal in the neuron group 

s (functional block group) 700 under consideration. When the answer of this decision is affirmative; the data 
on the output bus 305 of the first hierarchical level is sent out onto the output bus 315 of the second > 
hierarchical level. On the other hand, when the above decision results in the negative answer, the output of 
the tri-state buffer 156 is sent to the state of high resistance. By preparing a large number of the neuron 
groups (functional block groups) 700 each of the structure shown in Fig. 9 and connecting them through the 

10 buses 311 to 315 of the second hierarchical level to one another, it is possible to interconnect a greater 
number of the neurons (functional blocks) 100 without increasing excessively the capacities added to the 
individual buses. 

Fig. 10 is a schematic circuit for illustrating a method of interconnecting the buses of the hierarchical 
structure. In Fig. 10, reference numeral 701 to 706 denote the functional block groups (neuron groups) each 

is incorporating a plurality of the functional blocks (neurons) 100. The connection within each of the functional 
blocks as well as the interconnection thereof is made as shown in Figs. 2 and 7. In Fig. 10,, reference 
numerals 300, 302 and 303 denote a data bus (including an input bus 301 and an output bus 305), an 
address bus and a clock bus, respectively. Further, reference numerals 310, 312 and 313 denotes a high 
level data bus (input bus 311 and output bus 315), a high level address bus and a high level clock bus, 

20 respectively. Although the description of the embodiment now under consideration is based on the 
assumption that the cycle signal 304 is generated within each functional block (neuron) 100 on the basis of 
the clock signal (303) and the address signal (302), as in the case of embodiment shown in Fig. 3, it should 
be understood that the cycle signal bus can also be implemented in the hierarchical structure as illustrated 
in Fig. 9. Now turning to Fig. 10, the address signal and the clock are outputted onto the high level address 

25 bus 312 and the high level clock bus 313, respectively, from the timing generator 120 and supplied also to 
the address bus 302 and the clock bus 303 through drive circuit 722 and 723. respectively. (Each of the 
drive circuits 722 and 723 corresponds to the buffer 1 58 shown in Fig. 9.) In the case of the system shown 
in Fig. 10, different addresses are assigned to the individual function blocks (neurons), respectively, so that 
the functional block (neuron) can be definitely determined by the address allocated thereto. On the other 

30 hand, the data signal is coupled to the high level data bus 310 through the medium of bidirectional driver 
circuits 751 to 756 (each corresponding to the buffer 158 or 156 shown in Fig. 9) which are controlled by 
the address signal, wherein only the bidirectional drive circuit (one of the circuits 751 to 756) constituting a 
pair with the functional block group or neuron group (one of the groups 701 to 706) which includes the 
functional block (neuron) 100 designated by the address signal transkits the signal on the low level data bus 

35 300 onto the high level data bus 310, while the other bidirectional drive circuits transmit the signal from the 
high level data bus 310 onto the low level data bus 300. Accordingly, also in the case of the embodiment 
shown in Fig. 10, this output of the functional block (neuron) designated by the address signal is transmitted 
simultaneously to all the functional blocks (i.e. all the neurons). Incidentally, when the number of the 
functional blocks (neurons) is to be further increased, this can be accomplished by providing a functional 

40 block group (neuron group) corresponding to the whole circuit structure shown in Fig. 10 (except for the 
timing generator 120) as a structure of hierarchically high level together with higher level data bus, address 
bus and clock bus. By deepening the hierarchical structure in this manner, it is possible to increase the 
number of the functioanl blocks (neurons) while maintaining the upper limit value of the load connected to 
the data bus and others lower than a predetermined value. The control of the bidirectional drive circuit in 

45 the manner described above can be realized, for example, by determining the address signal such that the 
more significant bits thereof designate the functional block group (neuron group) with the less significant 
bits designating the individual functional blocks (neurons), wherein only the more significant bits are applied 
to address decoder such as that shown in Fig. 4 to obtain the control signal. 

It is noted that when the number of the functional blocks or neuron is increased, there arises a further 

so problem that the time involved in the processing increases. More specifically, in the system according to 
the invention, calculation progresses one step every time the address signal makes a round. Needless to 
say. as the number of the functional blocks or neurons increases, the upper limit value of the address signal 
increases correspondingly. This is turn means that the period taken for the address signal to make a round 
is correspondingly extended. This problem can not be evaded when all the functional blocks or neurons of 

55 concern are to be connected with all other functional blocks or neuron. In this conjunction, it is however 
noted that in many of practical applications, limitation of the range of the interconnection to a certain extent 
is not attended with problems of any significance. By way of example, let's suppose that temperature 
changes at individual portions or points of an object are to be determined by solving the equations of 
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thermal diffusion. In that case, although the instantaneous change in temperature at the individual points on 
the object is subjected to influence of the temperature at the point in the vicinity, it is substantially 
insusceptible to the direct influence of the temperatures at the remote points. In general, the instantaneous 
change in many physical quantities such as. for example, propagation of the electromagnetic wave scarely 
undergoes the direct influence of the situations prevailing at remote points but is effected only by the state 
or situation at points located in the vicinity of the point of concern. Accordingly, when the system according 
to the present invention is to be operated as an analogue computer, there arises practically no problems so 
long as each functional block can be connected with the functional block located logically in the 
neighborhood of the former. Fig. 1 1 shows a neuron network of an improved structure according to another 
embodiment of the invention in which the processing time is shortened in consideration of the facts 
mentioned above. 

Now referring to Fig. 11, reference numerals 701 to 706 denote, respectively, the functional block 
groups or neuron groups of the structure similar to those shown in Figs. 9 and 10. The address signal and 
the clock signals are supplied from the timing generator 120 to a high level address bus 312 and a high 
level clock bus 313. respectively, and hence to an address bus 302 and a clock bus 303 through driver 
circuits 722 and 723, respectively, as in the case of the systems shown in Figs. 9 and 10. Difference of the 
structure shown in Fig. 11 from those shown in Figs. 9 and 10 can be seen in the connection of the data 
bus 300. More specifically, each of the data buses 300 is interconnected with only both the adjacent data 
buses through the bidirectional driver circuits 851 to 856. wherein the bidirectional driver circuit transmits 
the signal from the functional block group or neuron group including the functional block or neuron 
designated by the address signal only to the two adjacent functional block groups or neuron groups. By 
way of example, let's suppose that the functional block or neuron 100 indicated by hatching in the functional 
block group or neuron group 702 of the neuron network shown in Fig. 11 is designated by the address 
signal. Then, the bidirectional driver circuit 852 transmits the signal from the functional block group (neuron 
group) 702 to the functional block group (neuron group) 852. while the bidirectional driver circuit 853 
transmits the signal from the functional block group (neuron group) 702 to the functional block group 
(neuron group) 703. On the other hand, the bidirectional driver circuits 851 and 854 transmit no signal in 
either direction. The control mentioned above can be achieved by constituting each of the driver circuits 
851 to 856 by a pair of buffer circuits of tri-state outputs connected together on the input side and the 
output side. In this supposed case, the output of the functional block or neuron designated by the address 
signal is transmitted to the individual functional blocks or neurons incorporated in the functional block 
groups (neuron groups) 701, 702 and 703 but not to those of the functional block groups (neuron groups) 
704, 705 and 706. Again, assuming that one of the functional blocks or neuron in the functional block group 
(neuron group) 705 (e.g. the functional block or neuron indicated by batching in the functional block group 
or neuron group 705 shown in Fig. 11) is allocated with the same address as that mentioned above, the 
output of that functional block or neuron can be transmitted to the individual functional blocks or neurons 
included in the functional block groups (neuron groups) 704, 705 and 706 by the same address signal as 
that mentioned above. Similarly, by assigning a same address to one of the functional blocks or neurons 
included in the functional block group (neuron group) 701 and to one of the functional blocks or neurons in 
the functional block group (neuron group) 704, the output of the functional block (neuron) in the functional 
block group (neuron group) 701 can be transmitted to the functional blocks (neurons) included in the 
functional block groups (neuron groups) 701, 702 and 706, respectively, while the output of the funct.onal 
block (neuron) included in the functional block group (neuron group) 704 can be transmitted to the 
individual functional blocks (neurons) of the functional block groups (neuron groups) 703. 704 and 705, 
respectively Similarly, by assigning a same address to one of the functional blocks (neurons) included in 
the functional block group (neuron group) 703 and one of the functional blocks (neurons) in the functional 
block group (neuron group) 706. signal transmission can take place among the functional block groups 
(neuron groups) 705. 706 and 701 as well as in parallel with the signal transmission taking place among the 
functional block groups (neuron groups) 702, 703 and 704. By virtue of the arrangement described above, 
the upper limit value of the address signal is determined to be constantly thrice as large as the number of 
the functional blocks or neurons included in the single functional group (neuron group) regardless of the 
total number of the functional blocks or neurons in the neural network. Consequently, the time taken for the 
address signal to make a round can remain constant independent of the total number of the functioani 
blocks or neurons. In the foregoing, it has been assumed that the data is to be transmitted only to the 
adjacent functional block groups (neuron groups). It should however be understood that data transmission 
can be made to take place not only to the immediately adjacent functional block groups (neuron groups) but 
also to those groups next thereto and hence among five functional block groups in total by correspondingly 
modifying the method of controlling the bidirectional driver circuits 851 to 856. In this case, although the 
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upper limit value of the address signal is five times as large as the number of the functional blocks 
(neurons) included in the single functional block group (neuron group), the number of the functional blocks 
or neurons to which data can directly be transmitted from the single functional block or neuron can be 
increased by a factor of "5/3 H . When it is desired to increase the number of the functional blocks or 

s neurons included in one functional block group (neuron group), this can be achieved by replacing each of 
the functional block groups (neuron groups) 701 to 706 by the whole system shown in Fig. 10 (exclusive of 
the timing generator 120) which is then regarded to be one functional block group (neuron group). 

In the structure shown in Fig. 11, the functional blocks (neurons) located one-dimensionaliy close to one 
another are interconnected. There can however be conceived such arrangement in which the functional 

io blocks or neurons located two-dimensionally close to one another are interconnected. Fig. 12 is a schematic 
diagram showing an improved structure of the neural network in which functional blocks (neurons) located 
two-dimensionally close to one another are interconnected. It should be noted in connection with Fig. 12 
that only the frames of the functional block groups (neuron groups) are shown together with the data bus 
and the bidirectional driver circuits with details of the internal structure of the functional block group (neuron 

75 group), the address bus, the clock bus and the timing generator being omitted from illustration, being 
understood that they are similar to those described hereinbefore by reference to Figs. 9, 10 and 11. The 
control of the bidirectional driver circuits in the neuron network shown in Fig. 12 is performed in such a 
manner that when the address of a functional block or neuron included in a given one of the functional block 
groups (neuron groups) is designated, data is transmitted not only from the column including that given 

20 functional block group (neuron group) to the adjacent columns but also from the row including that given 
functional block group (neuron group) to the adjacent rows. More specifically, referring to Fig. 12 and 
assuming, by way of example, that one of the functional blocks or neurons included in the functional block 
group (neuron groups) 705 shown in Fig. 12 is designated, data is transmitted to the functional blocks 
(neurons) included in the functional block groups (neuron groups) 701 to 709, respectively. By virtue of this 

25 arrangement, it is possible to interconnect the functional blocks or neurons which are two-dimensionally 
located close to one another. 

Next, description will be made of a manner in which the information processing system according to the 
invention is integrated in the form of a wafer scale integration semiconductor circuit (hereinafter also 
re f errec j to as WSI in abbreviation). With the phrase "the wafer scale integration semiconductor circuit or 

30 WSI circuit", it is contemplated to mean a semiconductor integration circuit integrated over a whole surface 
of the wafer size which can assure a higher degree of integration when compared with chip-like integration 
circuit obtained by cutting the wafer. 

Fig. 13 is a schematic view showing the information processing apparatus according to the invention 
which is realized in a single semiconductor substrate 500 in the form of the WSI (refer scale integration 

35 circuit). Referring to Fig. 13, reference numerals 700 denote the functional block groups or neuron groups 
described hereinbefore by reference to Figs. 9 to 12, wherein sixty functional block groups or neuron 
groups are shown in Fig. 13. It is assumed, by way of example, that each of the neuron groups (functional 
block groups) is assumed to include nine neurons (functional blocks) and thus 540 of neurons (functional 
blocks) in total are integrated in a single wafer. Referring to Fig. 13, reference numerals 311 to 315 denote 

40 buses formed horizontally and having the second hierarchical level. A plurality of the buses are provided 
over the whole surface of the substrate. Reference numerals 321 to 325 denote buses of a third hierarchical 
level formed in the vertical direction. It will be noted that only one set of the vertical buses is provided at a 
center region of the substrate. In the case of the instant embodiment, the neuron groups (functional blocks) 
700 are interconnected by the buses 311 to 315 of the second hierarchical level in a number ranging from 

45 two to four, wherein the buses 311 to 315 of the second hierarchical level are connected to buses 321 to 
325 of a third hierarchical level by way of a buffer part 350. The buffer part 350 has a structure similar to 
the connection between the buses 301 to 305 of the first hierarchical level and the buses 311 to 315 of the 
second hierarchical level and includes the input buffer, the decoder and the output buffer, as in the case of 
the structures shown in Figs. 9 to 12. At one ends of the buses 321 to 325, there is provided the control 

so apparatus 200 which includes the timing generator 120 and the function transformation circuit shown in Fig. 
6A or 6B. With this arrangement, it is possible to realize a neural network in which all the neurons 
(functional blocks) are interconnected while suppressing the maximum bus length to the diameter of 
substrate at the most. Concerning the power supply source, a power supply bus (not shown) can be so 
formed as to run in parallel with the buses 311 to 315 of the second hierarchical level so that the power can 

55 be supplied form power source parts 450 provided at both ends, thereby a large number of power source 
points can be assured. 

The wiring conductors to be used for the power supply bus (not shown), the buses 311 to 315 of the 
second hierarchical level, the buses 321 to 325 of the third hierarchical level should be greater in width than 
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the wiring conductors 301 to 305 used internally of the neuron groups (functional block groups) 700 and the 
buffer 350 should be wired with a greater space. This is because care must be taken so that these buses 
311 to 315 and 321 to 325 of great lengths should not present high wiring resistance and that the fault ratio 
should be lowered since otherwise occurrence of faults such a breakage, short-circuit and others in the 
5 buses of high hierarchical level will make inoperative many neurons (functioanl blocks) connected to these 
buses. For the failure which exerts influence to only a small number of the neurons (functioanl blocks), it is 
possible to exclude the neurons susceptible to the influenced of failure by setting the weight values thereof 
to zero on the side of the normal neurons (functional blocks). 

Further, it is also possible to provide a buffer circuit (not shown) on the wafer 500 at the position of the 
70 control apparatus 200 in place thereof, wherein the buffer is connected to the control apparatus 200 
provided externally of the wafer. When a fault should occur in the control apparatus 200. the whole system 
- is rendered inoperative. Accordingly, by providing the control apparatus 200 externally of the wafer 500 as 
-* described above, the control apparatus suffering a fault can easily be replaced or repaired. Besides, the 
fault ratio can be decreased by multiplexing the buffer circuit for the external connection or for connection 
is of the control apparatus. Alternatively, the buffer circuit may be provided at a center portion where tu yield 
is expected to be high. Further, a plurality of substrates 500 may be provided and connected to buses of 
r- the fourth hierarchical level by way of the buffer circuit mentioned above to thereby interconnect the 
i> plurality of wafers 500 for the purpose of implementing a neural network of a larger scale, as will be 
'x- described later on. 

-so When the number of the hierarchical levels for the buses is increased, the delay time involved in the 
transmission of the signal (data) produced by a terminal one of the neurons (functional blocks) 100 to 
another terminal neuron (functional block) 100 by way of the control apparatus 200 will be increased. In this 
conjunction, it should be mentioned that by inserting a latch circuit in the circuitry including the buffer for 
connecting the buses of the high and low hierarchical levels for synchronizing the signal transmission, a 
25 pipeline processing can be performed to prevent the processing time from being increased. An example of 
the timing control to this end will be elucidated in detail by reference to Fig. 14 on the assumption that the 
number of the neurons (functional blocks) is one hundred for simplification of the description. For the clock 
signal it is preferred to use a circuit assuring as high a transmission rate as possible so that the clock 
signal' can travel from the bus 323 of the third hierarchical level to the bus 313 of the second hierarchical 
ao level and thence to the bus 303 of the first hierarchical level with a shortest possible propagation delay 
time All the other signals are transmitted in synchronism with the clock signals of the relevant hierarchical 
levels The address signal is supplied in such a manner in which the addresses "1" to "100" required for 
designating the individual neurons (functional blocks) and the addresses "101" to "105" required for the 
time division control are repeated with an interval corresponding to one clock being interposed upon return 
of the maximum address value "105" to the minimum address value "1". For transmission in synchronism 
with the clock signal, the addresses are shifted by one clock upon propagation from the second hierarchical 
level to the first hierarchical level. Since the individual neurons (individual functional blocks) are connected 
with the address bus 302 of the first hierarchical level (refer to Figs. 3, 7 and 9). the output value of the 
neuron (functional block) 100 designated by the address on the bus 302 is outputted onto the output bus 
305 of the first hierarchical level. At this time, the abovementioned output signal is sent to the output bus 
* 315 of the second hierarchical level and thence to the output bus 325 of the third hierarchical level white 
«■ being shifted one by one dock in synchronism with the clock signal. The output signal is applied to the 
^ input bus 321 via the control circuit 200. In that case, when synchronization with the clock signal is realized 
within the control circuit 200. the signal is shifted by one clock. The signal is then sent to the input bus 31 1 
45 of the second hierarchical level and thence to the input bus 301 of the first hierarchical level while being 
shifted again one by one clock. In this way. after the time lapse corresponding to five clocks from the arrival 
of the address signal at the address bus 302 of the first hierarchical level, the signal from the neuron 
(functional block) 100 designated by that address signal can reach all the neurons (functional blocks). Thus, 
in each of the neurons (functioanl blocks), the cumulative addition is performed for the signal by regarding 
so that the signal originates in the neuron (functional block) of the address which is smaller by five than the 
numerical value indicated by the address signal. By virtue of the arrangement described above, the period 
of the clocks signal can be shortened when compared with the case where no synchronism is realized in 

the inter-bus transmission. . cnn 

When the integrated circuit is to be formed over the whole surface of the semiconductor substrate 500 
55 having the diameter of as large as several inches as shown in Fig. 13. resolution power will inevitably be 
degraded when irradiation for exposure is to be performed all at once in a step of forming a pattern through 
irradiation of a photoresist layer applied over the substrate 500. Accordingly, it is a general practice to 
expose the substrate divisionally through several irradiation steps by selecting one neuron group (functional 
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block group) as one unit for the exposure. In this connection, it is noted that when the patterns of the 
individual neuron groups (functional block groups) should differ from one another even only a bit, there 
arises the necessity to prepare the masks for all the patterns or the neuron groups. This means that an 
enormous number of different mask must be prepared which is too expensive and uneconomical. On the 

s other hand, the individual neurons (functional blocks) are assigned with the addresses determined specifi- 
cally for the neurons, respectively, and each necessitates the function for discrirninatively identifying the , 
specific address assigned thereto. In other words, it is necessary to be capable of discrirninatively 
identifying the individually specific addresses without changing or altering at all the internal structures of the 
individual neuron groups (functional block groups). Fig. 15 is a schematic circuit diagram showing a 

10 structure of the neural network which can dispose of successfully the problems mentioned above. 

Fig. 15 shows a right half of one row of the neuron groups (functional block groups) shown in Fig. 13. In 
Fig, 15, reference numeral 700 denotes generally the neuron groups (functional block groups), respectively. 
Further, a numeral 100 denotes a neuron or a functional block, numerals 311 to 315 denote buses of the 
second hierarchical level, numeral 156 denotes a tri-state buffer and a numeral 158 denotes a buffer circuit. 

is These constituents are interconnected in the manner illustrated in Fig. 9. A numeral 350 denotes a part for 
interconnecting the buses 321 to 325 of the third hierarchical level and the buses 311 to 315 of the second 
hierarchical level, numeral 166 denotes a tri-state buffer and 168 denotes a buffer circuit. It should be noted 
that the decoder 157 incorporated in each of the neuron groups (functional block groups) in the network 
shown in Fig. 9 is mounted in the connecting part 350 for interconnecting the buses of the third and second 

20 hierarchical levels. The output of the decoder 158 is sent to each of the neuron groups (functioanl block 
groups) 700 through a bus 316 of four bits. This bus 316 is so formed as to be shifted by one bit upon 
every passing through the neuron groups (functional block groups) 700, as can be seen in Fig. 15. The 
decoder 157 produces the output of high level only for the neuron groups (functional block groups) 700 that 
includes the neuron (functional block) 100 designated by the address signal. With such implementation, it is 

25 possible to select only the aimed one of the neuron groups (functional block groups) 700 notwithstanding 
that all the neuron groups (functional block groups) 700 are of the utterly same configuration. Each of the 
neurons 100 within each neuron group (each functioanl block group) can make decision as to whether the 
neuron itself is selected or not with the aid of the two less significant bits of the address signal and the 
signal sent from the decoder 1 57. Accordingly, the similar procedure can also be applied to the connection 

30 part 350 for selecting only one aimed row. In this case, the result of decoding the more significant bits of 
the address signal is sent to each of the connection parts 350 by way of the bus 326 of nine bits. Since the 
bus 326 of nine bits is so formed as to be shifted one by one bit for every row, it is possible to select the 
only one aimed row regardless of the utterly same configuration of the connection parts 350. According to 
the teaching of the invention described above by reference to Fig. 15, the number of the photo-masks 

35 required for forming the neuro-WSI shown in Fig. 13 can be decreased to five sets, i.e. one for the neuron 
groups (functional block groups) 700 of the right-hand side, another for the neuron groups (functional block 
groups) 700' of the left-hand side, another for the connection parts 350, another for the control circuit 200 or 
for the circuit for external connection and the other for the power supply pads 450. Parenthetically, it is 
noted that when the procedure mentioned above is adopted by using the semiconductor substrate 500 of a 

40 circular form, blank number addresses may take place. This problem can be coped with by skipping the 
blank number address when the address signal is sent from the control apparatus 200. In this manner, 
increasing in the processing time can be avoided, needless to say. Besides, instead of providing the 
decoders 107 in the neurons or the functional blocks 100 (refer to Rgs. 3 and 7), respectively, they may be 
provided en block or in a set in each neuron group (each functional block group) 700. In that case, the 

45 individual neurons (functional blocks) can be realized in the same circuit configuration, which apparently 
facilitates the circuit design. 

Next, description will be made of an improved information processing apparatus implemented in the 
WSI form by making use of the fault tolerance of the neural network and an excellent repeatability of the 
functional blocks (neurons). 

so Fig. 16 is a view showing an improved structure of the information processing apparatus integrated in 
the form of WSI according to yet another embodiment of the invention. Although the following description is 
based on the assumption that sixteen functional blocks (neurons) are integrated, the illustrated structure is 
never limited to any specific number of the functional blocks or neurons. In practical applications, five 
hundreds or more functional block (neurons) can be integrated on a single wafer 500. Referring to Fig. 16, 

55 1 00R and 100L denote the functional blocks (neurons), and numerals entered in the blocks represent the 
addresses of the individual functional blocks or neurons 100R and 100L. A reference numeral 350 denotes a 
buffer part having address translation means 90 (refer to Fig. 17) for translating the input address signal to. a 
signal differing therefrom for the output. A numeral 370 denotes an address input terminal to which the 
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address signal is inputted. Further, a numeral 200 denotes a control apparatus including an address pointer, 
and 38 designates an address pointer value outputted by the address pointer of the control apparatus 200. 
In Fig. 16, the addresses of the individual functional blocks (neurons) 100R and 100L at the time when the 
address pointer value 38 is "0000" are indicated by the numerals within the associated blocks, respectively. 
Concerning the details of the address signal translation means 90 and the individual functional blocks 
(neurons) 100R and 100L description will be made by reference to Figs. 17 and 18, respectively. 

Referring to Fig. 16, the number of the functional blocks (neurons) 100R and 100L is sixteen in total. 
Accordingly, for allocating the inherent or specific addresses to these neurons or functional blocks, four bits 
are sufficient for each functional block address. In the case of the example shown in Fig. 16, the two more 
significant bits of the four are used as the row address 3R with the two less significant bits being used as 
the column address 3L. Parenthetically, the bit number m for the actual address signal can be determined 
as follows: 

2 m1 < N £ 2 m (5) 

where N represents the number of the functional blocks or neurons. 
75 Fig. 17 shows the address translation means 90 incorporated in the buffer part 350. Although the buffer 
part 350 necessarily includes the buses for transmitting the clock signal and the data signal and the buffers, 
as described hereinbefore, they are omitted from the illustration in Fig. 16 for simplification with only the 
address signal translation means 90 being shown. This address translation means 90 corresponds to a part 
constituted by the address bus 326 and the decoder 157 shown in Fig. 15. In Fig. 17, a reference numeral 
370 denotes an input terminal for the address signal, 6R denotes a right-hand output, terminal for the 
address signal, 6L denotes a left-hand output terminal for the address signal. 6U denotes an upper output 
terminal for the address signal. 108 denotes a onerincrementor, and 110 denotes an inverter circuit for 
outputting the input logic signal after having inverted it. In the case of the instant embodiment of the 
address signal translation means or block 90, the row address 3R constituted by the two more significant bit 
25 of the address signal inputted through the address signal input terminal 370 is added with "1" by the one- 
incrementor 108 to be then outputted to the upper output terminal 6U for the address signal. The one- 
incrementor 108 has two bit inputs and two bit outputs and realizes the translations or conversions of "00" 
to "01" "01" to "10". "10" to "11" and "11" to "00". respectively. The other bits are outputted with the 
input values as they were. The address signal inputted through the address signal input terminal 370 is 
30 outputted intact from the right-hand output terminal 6R for the address signal 6R. Outputted from the left- 
hand output terminal 6L for the address signal is a signal which corresponds to the bit of the second least 
significance of the column address 3C of the address signal inputted through the address signal input 
terminal 370 and which has undergone the logical inversion in the inverter circuit 110. 

Figs. 18A and 18B show the functional blocks (neurons) 100L and 100R. respectively. A reference 
numeral 20 designates a functioanl circuit which is designed in dependence on the contemplated or 
intended application of the WSI. In the case of the WSI for the neuron computer, by way of example, the 
functional circuit 20 is so designed as to calculate neuron models and thus equivalent to the functional 
block or neuron 100 described in the foregoing. In Fig. 18A and 18B. arrangement concerning the address 
bus in the functional block 100 is shown with the other details being represented by the functional circuit 
20 in the figures, reference numerals 306 and 307 denote address signal input terminals, respectively, and 
308 and 309 denote address signal output terminals, respectively. The functional block 100L receives the 
address signal inputted through the right-hand address signal input terminal 306 and outputs through the 
left-hand address signal output terminal 308 a signal which corresponds to the least significant bit of the 
column address 3C of the inputted address signal and which has been logically inverted by the inverter 
45 circuit 110 The functional circuit 20 receives the address signal via the address signal input terminal 306. 
Similarly the functional block 100R also receives the address signal inputted via the left-hand address 
signal input terminal 307 and outputs via the right-hand address signal output terminal 309 a signal 
corresponding to the least significant bit of the column address 3C of the input address signal which has 
undergone logical inversion through the inverter circuit 110. The functional circuit 20 is inputted with the 
so address signal via the address signal input terminal 307. . , ^ . « nriD 

The four buffer parts 350 having the address signal translating function, the eight functional block 100R 
and the eight functional blocks 100L illustrated in Fig. 16 are connected in such a manner as mentioned 
below In the horizontal direction (i.e.- in the rowwise direction), the right-hand address signal output terminal 
6R of the address signal translation block 90 incorporated in the buffer part 350 and the address signal 
55 input terminal 307 of the functional blocks 100R are connected to, each other. The address signal output 
terminal 309 of this functional block 100R is connected to the address signal input terminal 307 of another 
functional block 100R. The left-hand address signal output terminal 6L of the address signal translation 
block 90 is connected to the address signal input terminal 306 of the functional block 100L The address 
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signal output terminal 308 of this functional block 100L is connected to the address signal input terminal 
306 of another functional block 100L. In the vertical (columnwise) direction, the address signal input terminal 
370 of the address signal translation block 90 is connected to the upper address signal output terminal 6U 
of the lower address signal translation block 90. Connected to the address signal input terminal 370 of the 

5 address signal translation block 90 located lowermost is the output of the address pointer provided in the 
control circuit 200. The functional block addresses of the individual functional blocks 100R and 100L 
connected in the manner described above are shown in Fig. 16 on the assumption that the address pointer 
value 38 outputted by the address pointer incorporated in the control circuit 200 is "0000". 

Fig. 19 shows the functional block addresses of the individual functional blocks 100R and 100L at the 

w time point when the address pointer value 38 is "0111". As will thus be seen, as the address pointer of the 
control apparatus 200 indicates various values, the separate input addresses inputted to the individual 
functional blocks 100R and 100L vary correspondingly in dependence on the address pointer values. 
Accordingly, when the functional block to which a predetermined address (e.g. "0000") is inputted is to be 
the selected functional block, it is possible to select rather arbitrarily the functional block by varying 

75 correspondingly the address pointer value 38. In the case of the example shown in Fig. 16, the functional 
block 100R located on the bottom row at the second column as counted from the right corresponds to the 
selected functional block. Further, in the case of the example illustration in Fig. 19 t the functional block 100L 
located on the topmost row at the first column as counted from the left is the selected functional block. 

Fig. 20 shows a functional block IOO'l corresponding to the functional block 100L shown in Fig. 18B in 

20 the case where more than three bits are required for the column address. In this functional block 100 L, the 
column address 3C of the address signal inputted via an address signal input terminal 342 is added with 
"1" by means of a one-incrementor 141, wherein the address signal resulting from the above translation or 
conversion is outputted to an address signal output terminal 343. The functional block 100 L can be realized 
by using the one-incrementor 141 in place of the inverter circuit 110 shown in Fig. 18. 

25 In the description of the embodiment made hereinabove, the address signal translation or conversion is 
effectuated by addition of "1". However, the address signal translation may equally be effected by 
subtraction of "1". 

By structurizing the functional blocks in such arrangement that the address signals follow paths of a 
tree-like configuration, as in the case of the structure shown in Fig. 16, breakage of wired conductors for the 

30 address signals in the functional blocks 100L and/or 100R or other fault will give rise to only a reduced 
number of the functional blocks which are rendered inoperative. By way of example, in the case of the 
structure shown in Fig. 16, occurrence of fault in any one of the functional blocks 100L brings about no 
more than two functional blocks which become inoperative at the most. 

On the contrary, in the case of a structure in which the sixteen functional blocks are connected in 

35 series, occurrence of a fault in one of the functional blocks makes inoperative all the functional blocks 
connected downstream of the block suffering the fault. 

In Figs. 16 to 20, the data bus and the clock bus for the communication among the functional blocks 
are omitted from illustration only for the sake of simplification. Further, although it has been assumed in the 
foregoing description that the number of the functional blocks provided repeatedly is sixteen, it will readily 

40 be understood that the invention is never limited to any specific number of the functional blocks, in order 
words, the structures described above can be applied to any desired number of the functional blocks by 
simple modifications which may readily occur to those skilled in the art without departing from the spirit and 
scope of the present invention. 

According to the prior art technique, realization of a large number of the functional blocks on a WSI with 

45 the specific or inherent addresses being assigned to the individual functional blocks, respectively, necessi- 
tates preparation of semiconductor reticle masks for wiring conductor layers separately for each of the 
functional blocks. When exposure or irradiation with one semiconductor reticle mask must be performed for 
each of the functional blocks/there will be required a great number of semiconductor reticle masks. In 
contrast, according to the teaching of the present invention incarnated in the illustrated exemplary 

so embodiments in which each of the functional blocks 100R and 100L is so arranged as to perform a 
predetermined translation on the input address to thereby output the address signal differing from the input 
address, each of the functional block is inputted with the address signal inherent thereto so that any given 
one of the functional blocks can be selected by the address assigned thereto, whereby the address 
decoders of the individual functional blocks can be implemented in the utterly sam structure. Accordingly. 

55 it is only necessary to prepare only three sheets of semiconductor reticle masks for the functional block 
100L, the functional block 100R and the buffer part 350 including the address signal translation block 90. 
respectively. Thus, the cost involved in manufacturing the semiconductor reticle masks can remarkably be 
reduced. 
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Further, in the case of the illustrated embodiments described above, there is adopted a dispersed 
control system in which the address decoding is carried out in each of the functional blocks. By virtue of 
this feature, the number of the signal lines can also be decreased considerably because of no necessity of 
providing a large number of the select signal lines for determining or selecting the functional block from one 
5 control circuit as with the case of a concentrated control system. 

As will be appreciated from the foregoing description, there has been provided according to an 
embodiment of the present invention a wafer-scale semiconductor integrated circuit of a structure realized 
by using the functional blocks in a repetitive pattern in which a first address signal is inputted to a first 
. functional block to undergo a predetermined translation, e.g. by addition or subtraction of "1" to or from at 
Jo least one bit. and a second address signal differing from the first address signal is generated to be 
outputted to a second functional block. In other words, in a WSI incorporating a plurality of functional blocks 
having a same function and implemented in a repetitive manner, the address signal generated externally of 
the wafer or by a circuit on the wafer is inputted to one part of the functional blocks, wherein each functional 
blocks translates the address signal inputted thereto to an address signal differing therefrom, which is the 
75 outputted to another functional block. By virtue of this arrangement, the individual functional blocks are 
inputted with the address signals differing from one another, respectively. When the address signal inputted 
n. to the one part of the functional blocks is modified or altered, the address signals inputted to. the other 
* functional blocks are also corresponding modified or altered. Accordingly, assuming that when a predeter- 
v mined signal is inputted to one of the functional blocks, that one functional block is selected, it is possible 
&o to select any given functional block by modifying the address signal inputted to a part of the functional 
blocks. To say in another way, it is possible to discriminative^ identify the functional blocks implemented in 
a repetitive pattern by controlling the address signal input The embodiment of the invention under 
consideration can be applied not only to the neuron computer WSI but also to the WSIs in general. 

It is noted that the number of the neurons (functional blocks) to be interconnected increases, the 
25 number of the weight values to be stored in each of the memories of the individual neurons (individual 
functional blocks) described hereinbefore is also increased, making it necessary to use the memory of a 
greater capacity to this end. Fig. 21 shows an improved structure of the weight value memory to be 
reduced. 

This memory circuit 101 is so structurized as to store only a predetermined number of the weight 
30 values orderly starting from the maximum absolute value in pairs with the counterparts 1 member's address, 
wherein the order weight values are regarded to be 0 (zero), to thereby reduce the storage capacity. 
Referring to Fig. 21. reference numerals 111 and 112 denote memory cell arrays, a numeral 113 denotes a 
pointer, a numeral 114 denotes a comparator and a reference numeral 115 denotes a selector. Of the 
memory cell array 111 and 112, the array 111 is destined for storing the counterparts' or members' 
35 addresses, and the array 112 is for storing the weight values, wherein the members' addresses and the 
respective weight values are stored in pairs in the memory cell arrays on a row basis. At the time, the order 
of the storage rowwise is so selected that the addresses are arrayed in the ascending order. Further, the 
address signals supplied from the control apparatus 200 to the individual neurons or functional blocks 100 
through the bus 302 and others are sent in the ascending order. The pointer 113 is set to indicate initially 
the first row of the memory cell arrays 111, 112. Then, the contents stored in the first row is read out, 
wherein the address information thereof is applied to one input terminal 122 of the comparator 114 while the 
weight value information is applied to one input terminal 124 of the selector 115. At this time point, the 
address information applied to the comparator is of the smallest value of the address information stored m 
the memory address cell array 111. On the other hand, the address signal incoming from the control 
apparatus 200 through the buffer via the address bus 302 is applied to the other input terminal of the 
comparator 114. Since the address signals are supplied in the ascending order of the address value, the 
address signal continues to be of a small value till the first occurrence of coincidence, with the signal 122 
applied to the comparator 114, which value is thus absent in the memory cell array 111. Consequently, the 
output 123 of the comparator 114 assumes a low level, as the result of which there makes appearance at 
the output terminal 126 of the selector 115 a numerical value of "0" which is applied to the other input 
terminal 125 of the selector 115. This output signal of the selector 115 represents the weight value for the 
counterpart member designated by the address signal 302 at that time point. In this case, the counterpart 
member's address is not stored in the memory cell array 111. In other words, the corresponding w ight is 
of the small absolute value which is regarded to be 0 (zero), as mentioned above. So long as the output 123 
of the comparator 1 14 is at low level, the output 127 of the AND circuit 1 16 is also constantly at a low level 
with the pointer 113 remaining in the state to indicate the same row. However, the coincidence occurs 
between the address information applied to the terminal 122 of the comparator 114 and the address signal 
sent via the address bus 302. the output 123 of the comparator 114 assumes a high level, which results in 
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that the weight value information applied to the terminal 124 of the selector 115 makes appearance at the 
output terminal 126 of that selector 115. This weight information represents the weight value for the 
counterpart designated by the address signal at that time point. In synchronism with the clock signal 
supplied via the bus 303, the pointer 113 is changed over to indicate the next or second row. Also in that 

5 case, the selector 115 continues to output "0* until coincidence occurs between the address information 
stored at the second row in the memory cell array 111 and the address signal supplied via the bus 302, 
whereon the similar operation to that described above is repeated. When the cycle signal is supplied via the 
bus 304, the pointer 113 regains the position to indicate first row again. As will now be appreciated, the 
circuit shown in Fig. 21 operates to read out accurately the weights of the large absolute values which are 

w stored in the memory cell array 112 while regarding the values of the other weights to be "0". Since the 
memory shown in Fig. 21 is necessarily read out in the ascending order, decision can instantaneously be 
made in response to the address signal as to the relevant information in present or not. Thus, the memory 
read-out operation can be performed without involving any appreciable increase in the access time. 

Fig. 22 is a block diagram showing in further detail a structure of the neuron 100 (functional block) in 

15 accordance with a further embodiment of the invention. Referring to Fig. 22, the memory 101 for storing the 
weight values, the multiplier 102, the adder 103, the latch circuits 104 and 105, the tri-state buffer 106, the 
decoder 107 and the buses 301 to 305 of the first hierarchical level operate in the manner described 
already by reference to Figs. 3 and 7. A shifter 807 and a selector 803 are provided for ensuring stabilized 
operation. Further, a decoder 801, a selector 802, a tri-state buffer 804, a latch circuit 805, selectors 806 

20 and 808 and buses 831 to 838 are provided to enable the neural network to make a self-learning. 

The shifter circuit 807 is so configured that the output value of the latch circuit 104 is shifted by a factor 
of 1/2 or 1/4 or 1/8 or so. The selector 803 is so implemented as to transmit ordinarily the output of the 
multiplier 102 to the adder 103 and to transmit the output of the shifter circuit 807 to the adder 103 in 
response to the application of the cycle signal (bus 304). Further, such arrangement is adopted that when 

25 the selector 803 transmits the output of the shifter circuit 807, the adder 103 performs subtracting operation. 
The selector 808 applies the output of the latch circuit 104 to the adder 103 except for the period of the 
self-learning mode. Thus, the adder 103 cooperates with the latch circuit 104 to perform ordinarily the 
cumulative addition described hereinbefore. However, when the cycle signal is applied, the output of the 
latch circuit 104 is fetched by the latch circuit and multiplied with 1/2 or 3/4 or 7/8 or so to be subsequently 

30 applied to the input of the latch circuit 104 and fetched by the latch circuit 104 in synchronism with the 
clock signal (supplied via the bus 303). whereupon a new cumulative addition is started. In this case, a 
weight mean of the value calculated in the preceding cycle and the value to be calculated in the instant 
cycle is arithmetically determined. Although a time is taken until convergence of the result, oscillation is 
difficult to take place. 

35 The self-learning operation is carried out in a manner described below on the assumption that algorithm 
for the self-learning is based on a back propagation (the mathematical foundation of which is described in 
detail, for example, in a Japanese periodical "Nikkei Electronics", published August 10. 1987, pp. 115 et 
seq.). In the first place, the cumulative adding operation is repeated to determine the output values of the 
neurons and store them in the respective latch circuits 105. Next, the addresses of the neurons belonging to 

40 the output layer are sent out sequentially from the control apparatus 200 via the address bus 302 to read 
the contents of the latch circuits 105 into the control apparatus for determining the difference from an 
expected output value (the value of this difference is hereinafter referred to as the 5-value), the difference 
value or 5-value is written in the latch circuit 805 of the relevant neuron via the bus 823. For the neurons 
which do not belong to the output layer, the signal on the address bus 302 is fixed to the address of the 

45 neuron for which the 5-value is to be determined, whereon the 5-value of concern is determined through 
cumulative addition of the products of the weight values of the individual neurons for the neuron of concern 
and the 6-value of the former. The weight value is read out from the memory 101 incorporated in each 
neuron with the 6-value being read out from the latch circuit 805. respectively, whereon both values are 
applied to the multiplier 102 of each neuron via the selectors 802 and 806. respectively. In this way. the 

so products are first determined. Subsequently, the addresses of the individual neurons are sent out from the 
control apparatus 200 via the bus 831, and the products mentioned above are read into the control 
apparatus 200 via the bus 832 to be cumulatively added for determining the 5-value which is then written in 
the latch circuit 805 of the neuron of concern. When the 6-values have thus been determined for all the 
neurons, the product of the 6-value stored in the latch circuit 805 and an appropriate coefficient supplied via 

55 the bus 837 is determined by means of the multiplier 102 to be subsequently applied to one input terminal 
of the adder 103. By updating a round the addresses applied via the address bus 302, the result obtained 
through addition of the weight value stored in the memory 101 and the value applied to the one input of the 
adder 103 is set as the updated or new weight value. By repeating the procedure described above, the self- 
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learning can be realized. 

Of course, it is also possible to perform the learning on another computer, wherein the weight value 
obtained thereby is written directly in the memory 101, so that only the ordinary operation is performed on 
the neural network. 

s As a factor contributing to increase the manufacturing cost of the WSI, there can be mentioned a 

problem of low yield rate. The measures for solving the problem in the WSI for the neuro-computer will be 
descussed below. By preparing a plurality of neurons and connecting them to one another, there can be 
implemented a variety of neural networks. By setting appropriate weight values (w j(j ) ( it is possible to 
execute various information processings. When the neural network of this type is implemented on a WSI. 

ro there may appear such situation in which some of the neuron circuits do not operate normally due to crystal 

: defect or presence of foreign materials. These faulty neurons can be disconnected by setting to zero the 
/■£ weight values of the neurons to which the outputs of the faulty neurons are inputted. Thus, the neural 
system as a whole can operate normally even when the fault or defect is present. Besides, since the neural 
network is inherently redundant and has a high fault tolerance, it is suited for implementation on WSI. 

75 It is however, noted that when the individual neurons are interconnected on the WSI by the bases, 
occurrence of fault in the bus and the address decoder makes it impossible to perform communications 

>: between the neurons. In other words, the yield rate of the WSI is degraded as a whole, involving high 
expensiveness thereof. In an extreme case, no product could be manufactured with a quality to be satisfied. 

^ Fig. 23 shows in a schematic circuit diagram an improve structure of the neural network which can enhance 

so the yield of WSI according to a further embodiment of the present invention. 

In Fig. 23. reference numeral 203 denotes multiplex clock signal lines, 351 denotes multiplex data 
buses. 100 denotes neurons (functional blocks), respectively. 203 denotes multiplex address signal input 
terminals, and 204 denotes multiplex address signal output terminals. The multiplex clock signal lines 355 
are realized by multiplexing the clock signal line 303 in the embodiment described previously. Similarly, the 

25 multiplex data buses 351 are realized by multiplexing the data bus 300 of the previously described 
embodiment. For the address buses, the decentralized or distributed control address system described 
hereinbefore by reference to Figs. 15 to 20 is adopted. The multiplex address buses are realized by 
mutliplexing the address signal line interconnecting the individual neurons (functional blocks). NumeraJ 201 
denotes multiplex-data bus connection terminals and 202 denote multiplex clock signal input terminals. The 

30 multiplex address signal input terminals 204 of a given one of the neurons are connected to the multiplex 
address signal input terminals 203 of the adjacent neuron (functional block) 100. The multiplex address 
signal input terminals 203 of the neuron (functional block) 100 located at the input end of the network are 
connected to the address pointer incorporated in the control apparatus 200. Of course, the address pointer 
can be provided internally of the wafer. The multiplex data bus connection terminals 201 of each neuron 

35 (functional block) 100 are connected to the multiplex data buses 351. The multiplex clock signal input 
terminals 202 of each neuron (functional block) 100 are connected to the multiplex clock signal lines 355. 
Parenthetically, such arrangement may equally be adopted that the buffer part 350 including the address 
signal translation block is provided at the center of the neurons (functional blocks) 100, as described 
hereinbefore in conjunction with Figs. 13 and 16. The structure illustrated in Fig. 23 is only for the purpose 

' 40 of facilitating the description. 

:jf Fig. 24A is a schematic circuit diagram showing an exemplary structure of the neuron (functional block) 

* 100 suite for the multiplex bus structure according to a further embodiment of the invention. Referring to 
Fig. 24A, a reference numeral 211 denotes a majority circuit, numeral 231 denotes a data input terminal, 
numeral 232 denotes a data output terminal, numeral 233 denotes a clock signal input terminal, numeral 
45 234 denotes address decoders, respectively, and numeral 212 denotes tri-state output buffers, respectively. 
Each of the tri-state output buffers has a signal input terminal for input signal, a signal output terminal for 
*r.: outputting an output signal and a control signal input terminal to which a control signal is applied, wherein 
>v the tri-state output buffer outputs the input signal to the output signal terminal or presents a high impedance 
r state at the output terminal in dependence on the control signal. Further, numeral 221 denotes address 
350 decoder input terminals, and 222 denotes an address decoder output terminal. The signals supplied via the 
multiplex data buses 351, the multiplex clock signal lines 355 and the multiplex address signal input 
terminals 303 shown in Fig. 23 undergo logically majority decision on a bit-by-bit basis in the associated 
majority circuits 211. respectively, the results of which majority decision are supplied to a data input 
terminal 231. a clock signal input terminal 233 and to an address signal input terminal 234. respectively, of 
55 a neuron model calculation unit 220. On the other hand, the output of the neuron model calculation unit 220 
is produced at the data output terminal 232 and outputted to the multiplex data bus connection terminals 
201 through three tri-state output buffers 212. Further, three signals inputted through the multiplex address 
signal input terminals 203 are supplied to three one-incrementors 108, respectively, to undergo address 
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translation, the results of which are outputted through the multiplex address signal output terminals 204. In 
this conjunction, when the neurons are connected in the general arrangement shown in .Fig. 23, it is 
sufficient to increment by 1 (one) only the column address. By way of example, when the output of the 
neuron (functional block) 100 is outputted onto the data bus 351 for the address signals all of "0°, the 
s address decoder 210 may be implemented in the form of a NOR circuit, as shown in Fig. 24B. Further, the 
control signal input terminals of the three tri-state output buffers 212 are connected in such a manner as 
shown in Fig. 24A. When the output value of the address decoder 210 is "true*, the associated trhstate 
output buffer 212 outputs the input signal, and otherwise presents the high-impedance state at the output 
terminal. 

70 The neuron model calculation unit 220 is implemented in the same structure as that of the functional 
block (neuron) 1 00 as described hereinbefore by reference to Figs. 3 and 7. More specifically, the neuron 
model calculation unit has a terminal 231 corresponding to the input bus 301, a terminal 232 corresponding 
to the output bus 305, a terminal 233 corresponding to the clock bus 303 and the cycle bus 304 and a 
terminal 234 corresponding to the address bus 302, respectively. 

75 The individual neurons 100 shown in Fig. 23 are selected one by one in correspondence with the 
address pointer values 38, respectively, as indicated by the address pointer to send out the output value 
onto the multiplex data buses 351. In the neuron 100 not selected, the signals of the multiplex data buses 
351 are subjected to the majority logic on the bit basis by the majority circuit 211 shown in Fig. 24, the 
result of the majority logic being outputted to the neuron model calculation unit 220. Similarly, the majority 

20 logic operation is performed in response to each of the clock signals supplied in multiplex via the multiplex 
clock signal line 355 shown in Fig. 23, whereon the results are outputted to the neuron model calculation 
unit 220. The address signal outputted by the control apparatus 200 shown in Fig. 23 undergoes the 
address translation in each of the neurons 100 to be outputted to the succeeding neuron 100. In the case of 
the illustrated embodiment, the address translation is effectuated by the one-incrementor 108. The signals 

25 before undergoing the address translation is subjected to the majority logic operation in the majority circuit 
21 1 on the bit basis, the results of which are outputted to the neuron model calculation unit 220. The output 
value of the neuron model calculation unit 220 produced at the data output terminal 232 is decided to be 
outputted or not onto the multiplex data buses 351 by the three tri-state output buffers 212 controlled by the 
address decoder 210. In the case of the structure as illustrated, the output of this neuron 100 is sent out 

30 onto the multiplex data bus 351 to be sent to the other neuron 100, when the address signals inputted are 
all "0". 

In case the WSI is to be manufactured with the aid of an electron beam drawing system without using 
the semiconductor reticlemask, the number of the functional blocks (neurons) to be designed can be 
decreased, which in turn means that the number of the design steps can be correspondingly decreased. 

35 In the following, multiplexing of the bus structure has been described. Of course, multiplexing of the 
learning circuit system is possible as well, as will be described below in conjunction with a back 
propagation method which is one of the learning procedures. 

Fig. 25 is a schematic diagram for illustrating the learning algorithm based on the back propagation 
method, in which one of the neurons (functional blocks) is shown. Reference symbols 11, 12 and 13 

4o designate the output signals of the individual neurons (functional blocks) of the preceding layer or stage, 
and a reference symbol OUT designates the output signal of the neuron (functional block) of concern, which 
signal is inputted to the synapses 1 1 of the neurons (functional blocks) of the succeeding layer or stage. In 
Fig. 25, solid lines indicate signal flows in the ordinary operation with broken lines indicating signal flows in 
the back propagation learning mode. The individual synapses 11 are imparted with the inherent weight 

45 values W, respectively, and each equivalent to a step where the input signal is multiplied with the weight 
value W to thereby output the product thereof. A reference numeral 10 represents a step for calculating the 
total sum of the input values multiplied with the weight values W. This step 10 corresponds to the functional 
circuit shown in Fig. 3. Further, the reference numeral 50 represents a step for outputting a value resulting 
from the conversion or translation of the input value in accordance with a function f. Parenthetically, as the 

so function used in the step 50, there is known the sigmoid . function and others. A reference numeral 60 
designates a switch which serves to change over the ordinary operation mode and the learning mode to 
each other. A reference numeral 70 represents a step for calculating a change d(n) of the synapse weight 
value in the learning mode. Reference symbol D(n + 1) designates learn signals from the succeeding 
stage. More specifically, relation between d(n) and D(n + 1) is given by the following expression (6): 

55 d(n) = a x f'(s) x I x total sum of D(n + 1) (6) 

where s represents the output of the step 10. £ represents a differential value of the function f at the step 
50 mentioned above, I represents the input signals (11-13) to the synapses, and d represents a constant. In 
Fig. 25, a reference numeral 80 represents a step for calculating the learn signal D(n) to be supplied to the 
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preceding stage, which can be calculated in accordance with 
D(n) = W x d(n) (7) 

By repeating the ordinary operation and the learning operation explained above, the weight values W of the 
synapses can be altered or changed in dependence on d(n), whereby the whole network can be 
s structurized as to realize the desired input/output correspondence. 

As can be seen from the above description, the pattern associative type network structure and the back 
propagation learning algorithm provide great advantage in that not only the simulation on the computer 
system but also implementation as a semiconductor integrated circuit can be much facilitated/ More 
specifically, in Fig. 25, it is easily possible to realize the part 11 in the form of a mutfiplier circuit with a 
70 memory circuit, the part 10 as an adder circuit the part 50 as a memory circuit, the part 60 as a switch 
circuit, the part 70 in the form of a combination of a multiplier circuit, an adder circuit and a memory circuit, 
and the part 80 as a multiplier circuit. 

On the other hand, difficulty may arise when some fault takes place in the learning circuit system. In 
that case, the neuron of the succeeding stage sets the weight value of the synapse to "0" by learning. 
75 However, because the faulty neuron transmits the erroneous learning signal to the neuron of the preceding 
stage, great difficulty is encountered in executing the correct learning. Accordingly, automatic remedy of the 
fault 'is impossible in the case of the fault mode mentioned above. Such being the circumstances, 
implementation in a semiconductor chip of the ordinary size of about 1 cm 2 encounters difficulty in attaining 
the yield approximating to 100%. Besides even when implemented in a wafer scale integrated circuit, 
improvement of the yield rate remains substantially equal to zero per cent, because the whole area 
occupied by the learning circuit system become very large. 

In view of the state of the art outlined above, there are provided according to still other embodiments of 
the present invention such improved structures in which the learning circuit system is separated from the 
individual neurons (functional blocks) and at the same time made common to the neurons, wherein the 
25 separated learning circuit system is constructed by an apparatus capable of remedying faults. 

Fig. 26 is a schematic diagram for illustrating the concept underlying the learning circuit according to 
the present invention. In Fig. 26. a reference numeral 99 denotes a learning circuit system which is 
provided separately from the individual neurons 100 and in common to them, and numerals 998 represent 
signal systems for the learning. A neural network shown as enclosed within a phantom-line block has the 
30 same structure as those described hereinbefore except that each of the neurons 100 has no learning circuit. 
The neural network enclosed by the broken line block executes normally the learning notwithstanding of 
occurrence of fault so long as the learning circuit system 999 is normal to thereby exclude the faulty 
neuron. Since the fault of the learning circuit according to the instant embodiment is previously remedied, 
no erroneously learned signal is transmitted to the neural network. The learning is executed, for example, tn 
35 the manner mentioned below. Namely, the synapse weight values of the neurons of a layer Lm are fetched 
by the learning circuit system 999 and rewritten into the original synapses after alternation of the weight 
value. This processing is executed for the neurons of the layer Lm one by one, whereon the similar 
processing is executed for the neurons of a layer Ln. 

Since all of the neurons (functional blocks) 100 are realized in the same circuit configuration, as 
described hereinbefore, it is possible to provide the learning circuit in common to the neurons. The learning 
circuit system 999 according to the instant embodiment of the invention is constituted only by the circuits 
11 10, 60, 70 and 80 and by a control circuit for allocating these circuits in common to the individual 
neurons or functional blocks/Accordingly, the learning circuit system can be implemented with a high 

integration density. x . . . 

Fig 27 is a diagram for illustrating an exemplary structure including the learning circuit system provided 
in common and a neuron 100 from which the learning circuit has been removed. Parenthetically, details of 
the functional blocks corresponding to the structure shown in this figure will be explained later on by 
reference to Figs. 28 and 29. Further, an exemplary embodiment of the fault remedy system for the 
learning circuit will be described hereinafter. In Fig. 27, reference numeral 99o denotes the learning circuit 
provided in common to the neurons (functional blocks), and 100 denotes a neuron from which the learning 
circuit has been removed. The synapse (combination of a multiplier circuit and a memory circuit 50 are 
equivalent to those shown in Fig. 25. A numeral 920 denotes a multiplication and addition circuit for 
multiplying the input signals 11 to In to the individual system synapses with the weight values W thereof and 
determining the total sum of the products resulting from the multiplication. This circuit 920 is equivalent to 
the circuit 11 and 10 described hereinbefore. A reference numeral 921 denotes a memory circuit for 
outputting a differential value of the function f described hereinbefore by reference to Fig. 25. A numeral 
922 denotes a multiplication and addition circuit for determining D(n) and d(n). as described hereinbefore by 
reference to Fig. 25. Circuit 921 and 922 are same as those designated by 70 and 80, respectively, in Fig. 
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25. The circuits 920, 921 and 922 are used in common to the individual neurons (functional blocks). A 
numeral 923 denotes a control circuit for controlling sequential allocation of the learning circuit to the 
individual neurons in a consolidated manner. The circuits 920. 921 and 922 are each of such a small scale 
as can be included in a single neuron shown in Fig. 25. The learning circuit 999 also requires as small an 

5 area as on the order of 0.5 cm 2 even in case the control circuit 923 is included. Accordingly, the yield rate 
of the learning circuit 999 is on the order of 60%. Since no fault is present in the network whose yield rate . 
is thus 100%, the yield of the semiconductor integrated circuit device as a whole is equal to the yield of 
60% of the learning circuit 999 provided separately from and is common to the neuron. By applying the 
instant embodiment to a wafer scale integrated circuit device, it is possible to increase the yield rate of the 

io wafer scale integrated circuit up to about 60% without need for the fault remedying structure. In other 
words, only by providing the learning circuit separately and in common, as described above, the yield can 
surprisingly be increased very significantly. In Fig. 27, reference symbols Ix and Wx represent wiring lines 
coupled to the weight values and the output values of the other neurons, respectively. 

Fig. 28 is a circuit diagram for illustrating details of the circuits 11. 10 and 50 shown in Fig. 27. 

75 Referring to Fig. 28, from the input signals II to 13 to the synapses 11 and the weight values W stored in 
the associated weight value registers 1111, products given by I x W are calculated by the multipliers 1100, 
respectively. The results of the multiplication are successively selected by a selector circuit 1112 to be 
added sequentially by an adder 1113. The result of the addition is inputted to a table I/O (input/output) 
circuit 1114, wherein a function value f corresponding to the result of the addition is selected from a f- 

20 function table (memory) 1115 and sent out as the output of the so-called cell. 

Fig. 29 is a circuit diagram for illustrating details of the circuits 920 to 923 shown in Fig. 27. Coupled to 
selector circuits 1010 and 1011 are the output signal 11 to In of the individual neurons and the weight values 
W1 to Wn of the individual synapses, respectively. Select signals 2000 and 2001 are outputted from 
address generating circuits 1000 and 1001 incorporated in the control circuit 923, respectively. The output 

25 signal of the neuron and the weight value selected by the selector circuit 1010 and 1011 are held by 
register circuits 1007 and 1012, wherein the product thereof given by I x W is determined by a multiplier 
1008. A numeral 1013 denotes a buffer circuit for writing in an reading out the synapse weight value. The 
results of the multiplications by the multiplier 1008 are successively added by an adder 1009. The result of 
this addition corresponds to s appearing in the expression (1). The result of addition s is inputted to a table 

30 I/O (input/output) circuit 10147 whereon a function f corresponding to the addition result s is selected from a 
\ -function table 1015. A multiplier 1017 is destined to execute the multiplication given by the expression (6) 
mentioned hereinbefore. The result d(n) of this multiplication is inputted to an adder 1016 to be added with 
the weight value W not updated and held in the weight value register 1012. The result of this addition 
represents the updated synapse weight value, i.e. a new synapse weight value after the learning. The 

35 updated value W is written into the individual synapse weight value registers (denoted by 111 in Fig. 28) 
incorporated in the network by way of the weight value register 1012, the read/write (RAW) buffer 1013 and 
the selector circuit 1011. A multiplier 1020 serves to determine the product of the value W held in the 
weight value register 1012 and d(n) in accordance with the expression (7) mentioned hereinbefore. The 
adder 1021 and the register 1022 cooperate to add sequentially the results of multiplication performed by 

40 the multiplier 1020 to thereby determine a total sum of D(n) and holds this determined value until the 
learning for the neurons belonging to the succeeding layer is executed, whereupon the value of D(n) is 
transferred to the register 1019. in the foregoing, description has been made of the structures and operation 
of the circuits 920, 921 and 922. A reference numeral 923 denotes a control circuit for controlling the 
learning process (weight value updating process) described above. The control is performed in synchronism 

45 with the clock signal inputted through a terminal 1005. Although it is shown in the figure that the clock 
signal is supplied externally of the control circuit 923, it goes without saying that the control circuit 923 may 
, incorporate therein a clock signal oscillator instead of being externally supplied with the clock signal. A 
numeral 1004 denotes a distributor circuit for supplying the clock signal to individual circuit blocks 
constituting the parts of the control circuit 923. By making use of this clock signal, a timing signal 

so generating circuit 1003 generates a timing signal. In accordance with this timing signal, a series of the 
learning processes, i.e. data transfer to the register, successive or sequential additions and multiplications 
are executed. Although it is shown in the figure that the timing signal 2000 is coupled only to the circuits 
920, 921 and 922, it will be understood that in reality the timing signal is supplied to the individual circuit 
blocks. A reference numeral 1002 denotes an I/O interface circuit for connection with the external circuits. 

55 An external control signal and other inputted through an input/output pin 1006 is decoded by the I/O circuit 
1002 which is turn sends control signals to a synapse address generating circuit 1000, a neuron output 
signal line address generating circuit 1001 and the timing signal generating circuit 1003. In accordance with 
the select signals sent from the synapse address generating circuit 1000 and the neuron output signal line 
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address generating circuit 1001, the selection of the values I and W described previously is executed. 

Fig. 30 is a diagram showing an exemplary structure for remedying fault of the learning circuit. In the 
figure, reference numeral 999' denotes a learning circuit provided separately and in common such as, for 
example, the one shown in Fig. 9. In the case of the instant embodiment, there are provided the learning 

5 circuits 999 in quintuplet. A numeral 997 denotes a switch circuit for selecting one of the outputs d(n) of 
the quintuple learning circuits 999 , After completion of the device manufacturing process, each of the 
leaning circuit 999 is inspected or checked as to fault or defeat, whereby the output of the learning circuit 
999' decided to be of no fault is selected by the switch circuit 997. When the yield rate of one learning 
^5 circuit 999' is 60%, as described previously, the probability of at least one of the quintuple learning circuits 

70- being faultless (i.e. the yield rate of the learning circuit) is 99%. It is possible to increase the yield by further 
y increasing the number of the learning circuits manufactured in multiplex. Incidentally, the size of one 
•z- learning circuit 999' is no larger than about 0.5 cm 2 . Accordingly, the area required by the five learning 
circuits as a while amount to no more than about 2.5 cm 2 and at the most 3 cm 2 inclusive of the switch 
circuit 997. The area is only about 5% of the device forming region of a 5-inch wafer. Fig. 31 is a schematic 

;s diagram showing a relation between the neural network and a learning circuit formed on a wafer according 
to an embodiment of the invention. In this figure, reference numeral denotes a learning circuit system 
t constituted by the learning circuits 999' provided in multiplicity and the switching circuit 997 described 
above by reference to Fig. 30. Numeral 100 denotes the neuron incorporating no learning circuit, as shown 
v in a lower half portion of Fig. 27. 

20:x In the above, the measures for remedying or coping with fault or detect of the learning circuit is 
described in conjunction with the structure in which the learning circuit or circuits are formed simultaneously 
with formation of the neurons on one and the same wafer. Further, it is also possible to remedy the fault of 
the leaning circuit by resorting to so-called hybrid means. Fig. 32 is a schematic diagram for illustrating, by 
way of example, a further means for coping with the fault of the learning circuit. In this figure, a reference 

25 numeral 500 denotes a wafer section in which a network of the neurons having no learning circuit is formed, 
and 800' denotes a semiconductor integrated circuit chip in section, which chip has a learning circuit 999^ 
formed therein. It should be noted that this learning circuit is not provided in multiplicity. The chip 800 
mounted on the wafer 500 is a faultless chip selected from a plurality of the chips 800 formed on a wafer, 
as shown in Fig. 33 and cut therefrom. The electrical connection of the wafer 500 and the chip 800 shown 

30 in Fig. 32 is realized by resorting to a flip-chip connecting method in which solder is employed as indicated 
by 30. The size of the chip 800' is 1 cm 2 . Fig. 34 is a schematic diagram showing still another exemplary 
embodiment for realizing a learning circuit 999 in a faultless structure. In the case of this embodiment, the 
circuit 999 is realized with the aid of a computer 550 which has already been ascertained to operate 
normally. A wafer 500 on which a neuron network is formed is mounted on a substrate 532. Connection 

35 between the substrate 532 and the wafer 500 is realized by wire bonding as indicated by 530, while 
electrical connection between the substrate 532 and the computer 550 is established by using a connector 
533 and a cable 534. Since the learning circuit 999 is implemented through cooperation ot the computer 
550 and a program running thereon according to the instant embodiment, the learning circuit system can 
...positively finished faultless. In the case of the instant embodiment, not only the signal for learning but also 

40-/ the input/output signals are transmitted from the computer 550 through the media 531, 532, 533 and 534. 

- a- The foregoing description has been made on the assumption that the chip or wafer is single. Fig. 35 is 

— a schematic diagram showing yet another embodiment of the invention in which , a plurality of the 
~ abovementioned wafers are stacked in the vertical direction. In this figure, numeral 500 denotes wafers each 

of which is packaged with an inventive learning circuit system 800 provided separately from and in common 
45 to the neurons. It should be noted that ten or more sheets of wafers 500 are stacked vertically. According to 
the instant embodiment, the wafers can be packaged very closely to one another, whereby a large scale 
-x, neuron network circuits can be packaged at an extremely high density. Although the learning circuit 800 is 
-. formed in each of the wafer, the former can be provided in common to plural wafers groupwise. Besides, 
the learning circuit 800 can also be realized by packaging the learning circuit chip 800 shown in Fig. 32 by 
50*- the hybrid method. Additionally, the learning circuit system may be connected to the computer 550 shown 
" in Rg. 34 by the connector 533 mounted on the substrate 532 for thereby effectuating the learning control 
with the aid of the computer 550. As a means for electrically interconnecting the wafers stacked in the 
vertical direction, there can be mentioned, for example, a method disclosed in JP-A-56-2662. 

In the case of the embodiments described so far, the information processing systems are wholly 
55 implemented by using the digital circuits. It is however noted that when only the multiplication and addition 
calculation or multiplication is performed by using an analogue circuit, the number of the elements which 
constitute one neuron can be decreased, whereby a large scale neural network can be realized. In case a 
large scale neural network is to be manufactured in which the weight values and the output values of the 
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neurons can be expressed by a multiplicity of bits, an enormous number of neuron circuits naturally involve 
an increased area for the packaging, even when the number of the elements constituting the single neuron 
circuit is decreased. Consequently, when the neural network is to be implemented, for example, in the WSI 
(wafer scale integration), the inter-neuron distance is increased. However, by performing digitally the inter- 

5 neuron data transfer, the data transfer over a great distance can be realized when compared with the case 
in which analogue signals are used for the communication. 

The following description is directed to the neurons according to other embodiments of the invention. 
A neuron constituted by a cellular model and a synapse model receives as inputs thereto the outputs of 
the others and assigns the weights to the latter, whereon the first mentioned neuron performs calculation 

;o mentioned below to produce an output value x f . Representing the outputs of N neurons of the identification 
numbers "0" to "N - 1 " by x 0 , xi Xj, .... x N .!, each neuron performs the calculation given by 



where a function f represents the sigmoid function given by 
*(S> =J/(1 + exp(S - S)/T)) (9) 

where s represents a threshold value and T represents a constant which is in general equal to 1 (one). 

In the case of the exemplary embodiments of the invention described hereinafter, the inter-neuron data 
transmission is performed by using digital signal, wherein the multiplication and addition calculation within 
each neuron is performed by resorting to analogue technique, the result of the analogue calculation being 
then converted to a digital signal. When the output of the analogue neuron circuit of digital interface is 
many-valued, the input/output is performed in terms of the digital signal more insusceptibly to noise as 
compared with the case of the analogue signal. Further, since the multiplication and addition calculation is 
performed by using the analogue technique, the calculation can be realized with a smaller number of 
neuron constituting elements when compared with the case in which the digital signals are processed. 

Fig. 36A is a schematic diagram showing an exemplary embodiment of the neuron in which the inter- 
neuron data transmission is performed by making use of the digital signal while the intra-neuron multiplica- 
tion and addition calculation is performed by the using analogue signals. In the case of the embodiment 
shown in Fig. 36A, the single neuron 100 is constituted by one cell-body or cellular circuit (functional circuit) 
10 and one synapse circuit (scalar circuit) 11. It should however be appreciated that the synapse circuit 
may be provided in any given number not smaller than one, inclusive. In this case, all the outputs of the 
synapse circuit 1 1 are connected to an input node 803 of the cell-body or cellular circuit (functional circuit) 

10. The output terminal 71 of the cellular circuit (functional circuit) 10 also constitutes the output of the 
neuron 100. 

The output of each of the other individual neurons 100 is coupled to an input terminal 61 of a digital 
PWM (pulse width modulation) conversion circuit 13 to be thereby converted to a pulse signal having a 
pulse width or duration proportional to the input value and outputted to an input node of the synapse circuit 

11. The synapse circuit 11 weights the outputs from the individual neurons 100. To this end, there is 
provided a weighting current generator circuit 501 which is supplied at the input the weight values wj, for the 
neuron outputs from a digital weight value output circuit 36, which may be constituted by a memory or 
register or the like, wherein the weight values w^ may previously be written in the memory or register. The 
weighting current generating circuit 501 includes a current source 701 connected to an internal node 802 of 
the synapse circuit 11, which current source 701 generates a current of magnitude proportional to the 
weight value. A switch 504 serves for changing over the internal node 802 and the input node of the cellular 
circuit (functional circuit) 10 between the conducting state and the non-conducting state. This switch 504 is 
controlled by the pulse control signal applied to the input node 51 from the digital PWM conversion circuit 
13. More specifically, the switch 504 sets the internal node 802 and the cell input node 803 to the 
conducting state only for a time corresponding to the pulse width of the pulse signal inputted to the input 
node 51. The current flowing into the cell input node 803 at that time is determined by the weighting current 
generating circuit 501 . Consequently, the amount of electric change flowing to the ceil input node 803 is in 
proportion to the product w ti , Xj of the neuron output value Xj and the weight value w tj therefore. A capacity 
507 is connected to the cell input node 803, whereby the potential at the cell input node 803 undergoes a 
change in proportion to the project Wjj, Xj. By performing the operation described above for the output 
values of the other neurons 100. the potential at the cell input node 803 undergoes a change which is given 
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Such potential change may be brought about simultaneously by a plurality of the synapse circuits 2 
connected to the cell input node 803. As described in detail hereinbefore in conjunction with the exemplary 
embodiments shown in Fig. 1 and Figs. 7 to 20, the multiplication and addition operation may be performed 
by utilizing the single synapse circuit 11 on a time-division basis. It should be mentioned that the capacity. 
507 is not the parasitic capacity but designed as mentioned below. Namely, representing by Qmax the 
maximum quantity of electric charge stored at the cell input node 803 by all the synapse circuits 11 
connected to the single cell input node 803. the capacity 507 is so selected as to satisfy the following 
condition. 

C = Q MAX /AMAX (10) 

Alternatively, the capacity 507 can be realized by corresponding wiring in the case of LSI or by making use 
of the gate capacity of an existing MOS FET or by providing additionally a MOS FET for setting the 
capacity 507. 

Subsequently, within the cell-body or cellular circuit (functional circuit) 10. an analogue-to-digital 
conversion (A/D conversion) and a non-linear transformation or conversion (in accordance with the sigmoid 
function) are carried out, the result of which is outputted to the output terminal in the form of the digital 
value More specifically, the potential at the cell input node 803 is converted to a digital signal by an 
analogue-to-digital converter 3, the digital signal being then outputted onto an internal data line 804 and 
subsequently inputted to a non-linear conversion circuit 26 to undergo the transformation in accordance with 
the sigmoid function, the result being finally outputted to the output terminal 71. 

According to the teaching of the invention incarnated in the embodiment described above, the 
multiplication and addition operation is realized by storage or withdrawal of electric change in or from the 
capacity 507, which operation can thus be realized with a low power consumption. Further, since the 
electric change is stored in the static capacity, the addition operation can be performed either in parallel or 
on a time-division basis. In this conjunction, let's suppose, by way of example, that the weighting is realized 
by resistors connected between the outputs of neuron circuits and the inputs of the succeeding neuron 
circuits, wherein the initial multiplication and addition operation by summing the currents flowing through the 
resistors. In that case, when the current flowing through the signal synapse circuit is assumed to be 100 uA 
on an average with the source voltage being 4 V. the power consumption all the synapse circuit will amount 
to as high as 100 W in case the number of the neuron circuits is 500 with each neuron circuit including 500 
synapse circuits. In contrast, in the case of the instant embodiment where the number of the neuron circuits 
is 500 each including 500 synapse circuits, the power consumption in all the synapse circuits .s as low as 5 
W when the current set by one weighting current generating circuit is 50 uA on an average, the time taken 
- for coupling the current to the capacity by way of the switch 504 is 20 ns on an average, the time taken for 
the calculation for one neuron circuit in accordance with the expression (8). is 200 ns. and the source 
voltage is 4 V. In other words, the power consumption can remarkably be reduced according to the teaching 

of the invention. . 

As will now be appreciated from the above description, there can be realized the neuron circuit in which 
the input/output operation is performed with the digital signal while the multiplication/addition operation is 
performed with the analogue quantities. 

- It should be added that in place of using the pulse signal having the pulse width proportional to the 
input value as the input signal to the synapse circuit 11. a pulse signal having a pulse density proportional 
to the input value may be utilized to the substantially same effect. 

Fig. 36B is a circuit diagram showing a modification of the embodiment shown in Fig. 36 A which is 
designed to be realized in CMOS LSI. 

In the description of operation which follows, the operation for raising the potential at a cell input node 
803 is referred to as -exciting" operation or "excitation" while the operation for lowering the potential at the 
cell input node 803 is referred to as "suppressing" op ration or "suppression". The excitation and 
suppression are controlled independence on a sign bit * s of the weight value outputted from a digital weight 
value output circuit 36. In Fig. 36B. it is assumed that the exciting operation is effective when the coded 
weight value bit <* s is at a potential WS (low level or "0") and the suppressing operation becomes effect.ve 
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when the coded weight value bit is a potential VDD (high level or "1"). 

Description is first directed to exemplary circuit configuration of the weight value correct generating 
circuit 501 and the switch 504. Source electrodes of a plurality of (e.g. four) PMOS FETs denoted by P, to 
P*. respectively, are connected to a power source VDD, drain electrodes thereof are connected to an 
internal node 810. and gates electrodes thereof are connected to sign bits m to respectively. Further 
provided are MOS FETs denoted by N1 to N*. respectively, which have respective source electrodes 
connected to a power source VSS. drain electrodes connected to an internal node 811 and gate electrodes 
connected to the sign bits *m to *>* weight value output circuit 36. Additionally provided is a PMOS FET 
denoted by Po which has a drain electrode connected to the cell input node 803. a source electrode 
connected to an internal node 810 and a gate electrode connected to an internal node 812. Also provided is 
a NMOS FET designated by N 0 which has a drain electrode connected to the cell input node 803, a source 
electrode connected to the internal node 811 and a gate electrode connected to an internal node 813. The 
switch circuit 504 is so designed that when the coded weight value bit a> s is at low level, the pulse signal 
inputted to the synapse input node 51 is converted to a pulse signal which turn on the PMOS FET P 0 the 
pulse signal being outputted to the internal node 812, while when the sign bit cj s at high level, the 
abovementioned pulse signal is converted to the pulse signal which turns on the NMOS FET N 0 , the pulse 
signal being then outputted to the internal node 813. To this end, there are provided an inverter circuit 22 
having the input applied with the sign bit w s of the weight value and outputs the logical inversion thereof, a 
two-input NAND circuit 24 for performing NAND operation on the outputs of the synapse circuit input node 
51 and the output of the inverter circuit 22. the result of the NAND operation being outputted to the internal 
node 812 and a Two-input AND circuit 24 for performing AND operation on the outputs of the synapse 
circuit input node and the sign bit *> s of the weight value, the result of which is outputted to the internal node 
813 

By setting the gate widths of the PMOS FETs Pi to P 2 and NMOS FETs N1 to N* in the order of the 
powers of "2 M , there can be realized sixteen levels of the weights. When representing conductances of the 
PMOS FETs Po to P* in the on-state are represented by G p0 to G p4 . then G p2 to G p4 are set as follows; 

Now representing the output bits of the digital weight value output circuit 36 connected to the gate of the 
PMOS FET Pt to P 2 by cm to a*, conductances between the VDD and the ceil input node 803 is given by 
the following expression (12) when the sign bit o s of the weight value is "0 n . Namely. 
1/(1/G p0 + 1(C p r(«i*2°+ 3 2 -2 1 + o> 3 -2 2 + o**2 3 )) (12) 
When 

G p0 »G p1 -(2° + 2 1 + 2 2 + 23) (13) . 
conductance between the VDD and the cell input node 803 can be approximated by the following 

expression: _ _ 

*> S -G P ,( «t*2°+ is-2 1 * « 3 *2 2 + a**2 3 ) (14) 

Thus, the gate width of the PMOS FET Po is set larger than those of the PMOS FETs Pi to P* so that the 
condition given by the expression (13) can be satisfied. Similarly, conductances of the NMOS FETs N 0 to 
N* are represented by G n o to G n4 . 
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G n2 * 21 * G nl 

G n3 = 2 2 • G nl ..... (15) 

G n4 " 23 * G nl 



20-^ 



By setting such that 

;G n0 » G n ,*(2° + 2 1 + 2 2 + 2 3 ) (16) 

the conductance between the cell circuit input node 803 and the VSS can be approximated by the following 
expression; 

<o s • G n1 <*m * 2° + o,2 * 2 1 + a> 3 # 2 2 + a* * 2 3 ) (1 7) 

By operating the PMOS FETs Pi to P* and the NMOS FETs Ni and N2 in the range of saturation, the 
weight value current generating circuit 501 serves as a current source capable of supplying the current of 
magnitude substantially in proportion to the weight value. Although the weight value is expressed in five bits 
inclusive of the sign bit in the case of the embodiment shown in Fig. 36B, it is of course possible to use 
'other number of bits to this end. 

After having set the weight value current generating circuit 501, the synapse circuit 11 applies the pulse 
signal to the input node 51 of the synapse circuit When the sign bit « s of the weight value is at a iow level, 
the PMOS FET Po is set to the on or conducting state for a period corresponding to the pulse width of the 
pulse signal representing the sign bit, while the NMOS FET N 0 is set to the on or conducting state for a 
period corresponding to the abovementioned pulse width when the weight value sign bit o> s is at high level, 
whereby the electric charge is injected into the cell input node 803. 

Next, description will be turned to the AD conversion circuit 3. At first, the up-down counter 81 is reset 
before operation of the synapse circuit 11. Further, the cell input node and the internal node 805 of the cell 
circuit 1 are precharged to the potential (VDD + VSS)/2 volts, respectively. Subsequently, the multiplication 
and addition operation described hereinbefore is performed, as the result of which the potential at the cell 
input node 803 undergoes a charge corresponding to the value resulting from the multiplication and addition 
operation. Upon completion of the arithmetic operation of the synapse circuit 11, the cell circuit 1 compares 
the potential V N at the cell input node 803 with the potential V R at the internal node 805 by using the 
comparator 20. The pulse control circuit 18 applies the clock signal to the clock signal input terminal 201 
and responds to the control signal outputted from the comparator 20 by applying the clock signal pulse to 
the internal node 2002 so that the PMOS FET Pi 0 is turned on when V N > V R while when V N < V R , the 
clock signal pulse is so converted that the FET Ni 0 is turned on to be subsequently applied to the internal 
node 2003. These are provided a PMOS FET Pto having a drain electrode connected to the internal node 
£805, a source electrode connected to the VDD and a gate electrode connected to the internal node 2002 
^and a NMOS FET N10 having a drain electrode connected to the internal node, 805, a source electrode 
connected to the VSS and a gate electrode connected to the internal node 2003, respectively. As a result, 
when a pulse of a constant pulse width is applied to the internal node 2002 or the internal node 2003 once 
or a plurality of times, the potential V R at the internal node undergoes a change proportional to the number 
of pulses as transmitted. Consequently, the potentials V N and V R become substantially equal to each other, 
resulting ultimately in inversion of the output of the comparator 20. At the time point, the pulse control 
circuit 18 stops transmission of the pulses. The pulses outputted from the pulse control circuit 18 is sent to 
the up-down counter 81 as well. The up-down counter 81 has the inputs connected to the internal node 
: 2002 and the internal node 2003, respectively. Wherein upon application of the pulse to the internal node 
.2002 which causes the PMOS FET P t0 to be turned on, the up-down counter counts up by +1, while the 
latter counts down by -1 upon application of the pulse to the internal node 2003 which causes the NMOS 
FET N 10 to be turned on. After having performed the operation described above for the data incoming from 
all the neuron circuits, the content of the up-down counter 81 then represents the result of the 
multiplication/addition operation. This result is then outputted onto an internal data line 804. 

It is also possible to arrange such that every time the synapse circuit 1 1 receives the output from each 
55 of the individual neuron circuits 100, the cell input node 803 is precharged by a half as described 
hereinbefore to thereby perform the multiplication operation. In this case, resting of the up-down counter 81 
is not effectuated and caused to operate as an accumulator. 

Next, a concrete circuit configuration of the non-linear transformation or conversion circuit will be 
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described in detail by reference to Figs. 37A and Fig. 37B in which Fig. 37A illustrates graphically values 
taken by the sigmoid function taken along the ordinate with the input values being taken along the abscissa, 
and Fig. 37B shows a memory map- 
Referring first to Fig. 37A, values of the sigmoid function to be outputted for the addresses as inputted 

s are written in the memory at the addresses corresponding to the values resulting from the multiplication and 
addition operation. The non-linear conversion circuit 26 internally contains the memory and receives at the 
input thereof the result of the multiplication and addition operation to thereby output the value written at the 
corresponding address to the output terminal 71 . The output is a digital value. 

A plurality of the neuron circuit 100 described above is mutually connected to constitute a neural 

w network, whereon the operation described in the foregoing is repeated. The arithmetic operation is executed 
until all the outputs of the neuron circuits 100 become stabilized (converge sufficiently). However, there 
arises sometimes such a case in which the outputs do not converge to the optimum solution (a so-called 
local minimum problem). In such case, convergence to the optimum solution may be attained by performing 
a simulated annealing, which can be realized by changing the constant T (temperature) in the sigmoid 

75 function given by the expression (9). Since the non-linear translation or conversion circuit 26 is of such 
structure that the functions are written in the memory, wherein the memory content at the address 
corresponding to the input value is outputted, the simulated annealing can be performed in a facilitated 
manner by rewriting the content of the memory which is constituted by a RAM (random access memory). 
According to the embodiment of the invention now under consideration, the weight value is handled in 

20 terms of the digital signal. In this conjunction, it may be conceived that the analogue signal is applied to the 
gate terminal of the MOS FET to control the conductance of the drain-source path of the MOS FET for 
thereby expressing the weight value and the input value in the form of the many-value. However, when the 
result of the arithmetic operation is to be outputted in the form of many-value, it is difficult to control the 
abovementioned conductance of the MOS FET by the gate voltage because of the non-linear characteristics 

25 of the MOS FET. 

In the case of the neural network according to the instant embodiment of the invention, the 
multiplication/addition operation within the neuron circuit is performed by making use of the analogue 
process, while the input/output operation of the neuron circuit is performed digitally. By virtue of such 
arrangement, the neuron circuit can be realized by a remarkably small number of constituent elements 
30 when compared with the neuron circuit implemented only by the digital circuitries. Further, since the 
input/output operation is performed by using the digital signal, there can be evaded such problems as noise 
margin and signal attenuation due to the wiring resistance to the analogue signal which may arise when the 
data transmission between the neuron circuits is performed by using the analogue signal, whereby design 
can be much simplified. 

35 Fig. 38 is a schematic circuit diagram showing a circuit configuration of the digital PWM converter 13. 
In this figure, a reference numeral 2101 denotes an input terminal for a conversion start signal, a numeral 
2102 denotes a clock signal input terminal, numerals 2103 to 2106 denote data input terminals, respec- 
tively, a numeral 2110 denotes a 4-bit counter, a numeral 2111 denotes a RS flip-flop, a numeral 2112 
denotes an AND circuit having four inputs, and a reference numeral 2110 denotes an inverter circuit. The 

40 counter 21 10 is decremented one by one every time the pulse of the clock signal is inputted from the node. 
By setting the data first to the input terminals 2103 to 2106 and then applying a pulse to the conversion 
start signal input terminal 2101, the output of the flip-flop 2111 becomes high. At the same time, 
transmission of the clock signal to the node 2102 is started. The data already placed in the counter 2110 is 
decremented one by one every time the clock signal pulse is inputted. When the counter ultimately outputs 

45 M 0 rt , the output of the 4-input AND circuit 2112 becomes high, where the output of the RS flip-flop 2111 is 
reset to low level. The output of the RS flip-flop 21 1 1 is maintained at high level for a period from the 
inputting of the conversion start signal to the inputting of a number of clock signal pulses having input 
values inputted to the input terminals 2103 to 2106. By executing the operation described above, the digital 
signal can be converted to the PWM signal. 

so Fig. 39 is a schematic circuit diagram showing a hop-field type neural network implemented by using N 
neuron circuits 100 according to another exemplary embodiment of the invention. Referring to Fig. 39, each 
of the neuron circuit 100 is constituted by one synapse circuit 11 and one cell body circuit 10. Every two of 
the neuron circuits 100 are combined in a pair, wherein the cell input notes 803a and 803b of the paired 
neuron circuits 100 are connected to a half precharge circuit 41 which serves to precharge the cell input 

55 nodes 803a and 803b to a voltage level of (VDD + VSS)/2 volts. All of the the output terminals of N neuron 
circuits 100 are connected to a time-division broadcast network 35. Representing the output values 
outputted from the output terminals 71 of the individual neuron circuit 4 by x 0 , x,, x 2 , x n .l the time- 
division broadcast network 35 selects one of these output values to thereby output the values xo to x M ., 
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sequentially in this order to the digital PWM converter 13. The digital PWM converter 13 the output value of 
the each neuron circuit 100 to a pulse signal having a pulse width or pulse density proportional to the input 
value (i.e. output of the neuron circuit) to thereby send the pulse signal to the input terminals 51 of all the 
neuron circuits 100. By performing N times the operation described above, all the data as required by the 
neuron circuit 100 arrive at the individual neuron circuits 100. Since the multiplication and addition operation 
is performed on a time division basis in the case of the embodiment shown in Fig. 39, one neuron circuit 
100 requires no more than one synapse circuit 11, whereby the amount of hardware can be reduced very 
significantly. 

Fig. 40A is a circuit diagram showing a configuration of the half precharge circuit 41, and Fig. 40B is a 
timing chart for illustrating operation of the half precharge circuit. In Fig. 40A, reference numerals 150 and 
151 denote clock signal input terminals, and 803a and 803b denote input terminals which are connected to 
the cell body input modes 803 two neuron circuits 100. respectively. Numerals 507a and 507b denote 
capacities connected to the cell body input nodes 803a and 803b t respectively. Reference numerals 71a 
and 71b denote the output terminals of the cell body circuit 10. in connection with the two cell body input 
nodes 803a and 803b, the precharge operation is performed in such a manner in which the clock signal 
input terminal 150 is set to a high level with the clock signal input terminal 151 being applied with a low 
level, whereby the- one eel! body input node 803a is charged to a high level with the other cell body input 
node 803b being at the low level. Subsequently, when the low level is applied to the clock signal input 
terminal 150 with the high level being applied to the clock signal input terminal 151, the PMOS FET P 2 i 
and NMOS FET N 2 i are turned to the on or conducting state, whereby a short-circuit is formed between the 
two cell body input nodes 803a and 803b. Thus, by realizing the capacities 507a and 507b substantially 
equal to each other, the two cell body input nodes 803a and 803b can be respectively half-precharged to 
(VDD + VSS)/2 volts. 

Fig. 41 is a schematic circuit diagram showing a hop-field type neural network which is realized by- 
using N neuron circuits 100 each constituted by one cell body (cellular) circuit 10 and (N - 1) synapse 
circuit 11. In one neuron circuit 100, the outputs of the all the synapse circuit 11 are coupled to one node of 
the cell body input nodes 803 (by wired-OR connection). Inputted to each synapse circuit 11 is the output 
from each neuron circuit 100. whereon the inputs is correspondingly weighted, and the amount of charge 

approximately proportional to the product is injected to each cell body input node 803 (803a - 803c 

803z). Thus, the multiplication and addition operations are performed in parallel by the individual neuron 
circuits 100. In the cell body circuit 10 of each neuron circuit 100, there are performed the AD conversion 
and the non-linear transformation, being followed by conversion to the pulse signal through the digital PWM 
conversion circuit 13, which pulse signal is then outputted to the input terminal 51 of the synapse circuit 11 

of each neuron circuit 100. The cell body input node 803 (803a to 803c 803z) of each neuron circuit 100 

is connected to the half-precharge circuit 41 such as the one shown in Fig. 39. 

The operation described above is repeated until all the output values become stable. 

In the case of the embodiment shown in Fig. 41 , the multiplication and addition operation as well as the 
non-linear transformation are performed in parallel with each other. Accordingly, although the amount of 
hardware is increased because of necessity of the synapse circuits about N times as many as those 
required by the network shown in Fig. 39, operation speed can be increased about N times as high. 

Fig. 42 is a schematic circuit diagram showing a hop-field type neural network similar to that shown in 
Fig. 41 but differs from the latter in that the output of each neuron circuit 100 is expressed by a single bit of 
"0" or "1". Because the output value of the neuron is expressed by the single bit, the cell body circuit 10 
can be simplified as compared with that described above. More specifically, an inverter circuit 21 Is 

connected to the cell body input node 803 (803a - 803c 803z), wherein two inputs of a 2-input AND 

circuit 24 are connected to the output terminal of the inverter circuit 21 and the clock signal input terminal 
202, the output terminal of the AND circuit 24 being connected to the synapse input node 51 of each 
synapse circuit 11. 

When the logical threshold value of the inverter 21 of the cell body circuit 10 is represented by Vthl, 
the output produced by each neuron circuit 1 00 can be given by 
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Each neuron circuit 100 produces the output only when the clock signal input terminal 202 is high. 

In the case of the embodiments shown in Figs. 41 and 42, the outputs of all the synapse circuits 1 1 
within the single neuron circuit 100 are connected through the wired-OR at the cell body input node 803. In 
this conjunction, it is noted that when an extremely great number of the synapse circuits, e.g. several 
hundred synapse circuits 11 are connected to one cell body input node 803, there are required wiring 
conductors each of a correspondingly increased length, as the result of which the signal may be attenuated 
due to the wire resistance and the leakage. Thus, there may take place such situation in which the signal 
can not arrive at the cell body input node or may undergo the influence of noise or be distinguished by 
noise. For coping with the problems mentioned above, it is further taught by the present invention to 
connect in a two-stage tree structure the four-input neuron circuits 100Q as the basic neuron circuits to 
thereby realize a 16-input neuron circuit 4T16 or a greater scale neuron circuit having a greater number of 
inputs by connecting the 16-input neuron circuits 4T16 in multiple stages, as shown in Fig. 43. Referring to 
the figure, reference numerals 1301 to 1316 denote input terminals of the synapse circuits. The structure 
shown in Fig. 43 may be implemented by using the basic neuron circuits each constituted by an eight-input 
neuron circuit or a two-input neuron circuit, as occasion requires. In this case, in each of the repeater cell 
body circuits 10 except for the cell body circuit 10 of the final output stage, a linear transformation 
(conversion) is performed rather than the non-linear transformation. 

Fig. 44 shows another embodiment of the present invention. The circuit is similar to the embodiment 
shown in Fig. 36B with regard to the synapse circuit 1 1 and differs slightly from the latter in respect to the 
structure of the cell body circuit. 

Referring to Fig. 44, reference symbol Ps designates a PMOS FET and Ns designates a NMOS FET. 

In the cell body circuit 1b, the cell body input node 803 is precharged to (VDD + VSS)/2 (V) by the 
half-precharge circuit (not shown), whereby an up-down counter 81 is reset. Further, there is provided a cell 
body input node for applying the potential of (VDD + VSS)/2 (V) is applied to the reference voltage input 
terminal 806. The PMOS FET P 5 has a drain electrode connected to the ceil body input node 803. a source 
electrode coupled to VDD and a gate electrode connected to the internal node 2004. On the other hand, the 
NMOS FET N 5 has a drain electrode connected to the cell body input node 803, a source electrode 
connected to the VSS and a gate electrode connected to the internal node 2005. The clock signal is applied 
to the clock signal input terminal 201, and the output of the comparator 20 is used as a control signal. More 
specifically, when the result of the comparison performed by the comparator 20 is that V N < N R2 , the clock 
signal pulse is applied to the internal node 2004 which sets the PMOS FET P 5 to the on-state, while when 
V N > Vr2. the pulse is outputted to the internal node 2005 for turning on the NMOS FET N 5 - This operation 
is controlled by a pulse control circuit 1 8. After the pulse is outputted to the internal node 2004 or 2005 
once or a plurality of times, V n becomes approximately equal to VR2, whereby the output of the comparator 
20 is ultimately inverted. At that time point, the pulse control circuit 18 stops the transmission of the pulse 
signal. An up-down counter 81 is connected to the internal node 2004 and the internal node 2005 for 
coupling "-1" in response to the application of the pulse to the internal node 2004 which turns on the PMOS 
FET P 5 . while counting " + 1 W in response to the application of the pulse to the internal node 2005 which 
turns on the NMOS FET N 5 . At the time when the above operation has been performed for the data from all 
the neuron circuits 100, the value of the up-down counter 81 indicates the results of the multiplication and 
addition operation, which is then subjected to non-linear transformation by the non-linear transformation 
circuit, the result of which is outputted to the output terminal 1 1 . 

Every time the output of each neuron circuit 100 is inputted, the half-precharge of the cell body input 
not 803 can be effectuated to thereby realize the multiplication operation. In that case, the up-down counter 
810 is not reset so that it operates as an accumulator. 

Fig. 45A shows an another cell body circuit 1c for realizing the AD conversion circuit and the non-linear 
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transformation circuit 26 according to a still further embodiment of the invention. Referring to Fig. 45. 
description will be made of operation which takes place in succession to the change in the potential at the 
cell body input node 803 as the result of multiplication/addition operation performed by the synapse circuit 
11. 

5 A comparator 20 is provided for comparing the potential V N at the eel! body input node 803 with a 
potential V R3 at a staircase wave output node 807 of a staircase wave generating circuit 25. Further 
provided is a counter 19 for counting the number of pulse outputted to a pulse output node 808 of the 
stircase wave generating circuit 25 under the control of the output signal of the comparator 20. The counter 
19 is reset before performing the AD conversion and non-linear transformation. Subsequently, the potential 

70 V N is compared with V R3 by the comparator 20. The counter 19 continues to count the number of pulses 
outputted to the pulse output node 808 until the output of the comparator 20 is reversed. The value 
indicated by the counter 19 upon reversion of the output of the comparator 20 represents the result of the 
non-linear transformation or conversion. For the non-linear transformation of the sigmoid function f, the 
staircase wave V R2 is outputted in the form of a inverse function r^k) of the sigmoid function f for the 

75 number k of the pulses outputted to the pulse output node 808. 

In the foregoing, the neural networks of the hop-field type have been described. It should be understood 
that a hierarchical-type neural network can also be realized. Fig. 46 shows an example of the neural network 
of a hierarchical structure. Referring to the figure, the neuron circuit 100 is constituted by the synapse 
circuits 11 and the eel! body circuit 10. Referring to Fig. 46, an input pattern inputted as indicated by 2001 

20 undergoes arithmetic operation through the neurons of an input layer 2100. an intermediate layer 2200 and 
an output layer 2300, the result of the operation being outputted through the neurons of the output layer 
2300, as indicated by 2002. 

25 Claims 

1. An information processing system, comprising: a plurality of blocks (100) each including a first circuit 
(101) having a storage function, a second circuit (102) having a multiplication function and a third circuit 
(103) having an addition function; each of said blocks being so arranged that products of a value stored in 
said first circuit (101) and outputs of the other blocks (100) are determined by said second circuit (102), and 
a total sum of said products is determined as an output value of said block, and that the blocks designated 
sequentially by an address signal (302) output the respective output signals simultaneously to the other 
blocks 

2. An information processing system according to claim 1, wherein a state is provided in which none of 
35 said blocks is designated in the course of a round of said address signal. 

3. An information processing system according to claim 1 or 2. further including a circuit (104) for 
storing temporarily the output of said third circuit (103). . 

4. An information processing system according to one of claims 1 to 3, further including a circuit (106) 
for storing temporarily the output of said block. 

40 5. An information processing system according to one of claims 1 to 4, wherein said plural.ty of the 
blocks (100) are -grouped into a plurality of groups (700), each of said groups being provided with a data 
bus (301 - 305) on a group basis for transmitting the output of each of the blocks to the other blocks, and 
wherein a data bus (311 - 315) of high level is provided for mutually connecting said data bus provided on 
the group basis. 

6 An information processing system according to one of claims 1 to 5. wherein at least two of said 
plurality of the blocks (100) output the signals simultaneously, the output signal of each of said at least two 
blocks is transmitted to only a part of said plural blocks. . ^ 

7 An information processing system according to one of claims 1 to 6. further including an address bus 
(302) for transmitting said address signal and an output bus (305) for transmission of the output signal of 
each of said blocks (100). wherein each of said blocks being so arranged as to send out the output signal 
onto said output bus only when said block is specified by said address signal (302). 

8 An information processing system according to one of claims 1 to 7. wherein said plurality of the 
blocks (100) are grouped into a plurality of groups (700). further including a bus (311 - 315) of a first 
hierarchical level provided for each of said groups and connected to the blocks belonging to the associated 
group, and a bus (321 - 325) of a second hierarchical level provided . in common to a plurality of said groups 
(700) and connected to the buses (31 1-315) of said first hierarchical level of said plural groups. 

9. An information processing system according to claim 8. wherein the bus (321 - 325) of sad second 
hierarchical level is wired by using a conductor having a greater width than that for the bus of said first 
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hierarchical level- 

10. An information processing system according to claim 8 or 9, further including means (120) for 
generating a clock signal, wherein signal transmission between the bus of said first hierarchical level and 
the bus of said second hierarchical level is performed in synchronism with said clock signal. 

5 11. An information processing system according to one of claims 1 to 10, wherein each of said blocks 

(100) includes address translation means (90) for performing a predetermined translation for the address 
signal inputted to thereby output and address signal differing from said inputted address signal. 

12. An information processing system according to claim 11, wherein said address translation means is 
constituted by means (108) for adding or subtracting at least one bit to or from said inputted address signal. 

w 13. An information processing system according to claim 11 or 12, wherein each of said blocks (100) 
includes an address decoder (107) for responding to the address signal which is same as a predetermined 
signal. 

14. An information processing system according to one of claims 1 to 14, wherein said first circuit (101) 
stores therein a predetermined number of weight values which is smaller than that of said blocks in the 

75 order starting from the maximum absolute value of said weight 

15. An information processing system according to one of claims 1 to 14, wherein said system is used 
as a neural net. 

16. An information processing system according to one of claims 1 to 15, wherein said first, second and 
third circuits (101, 102, 103) are constituted by digital circuits, respectively. 

20 17. An information processing system according to claim 14, wherein said first circuit (101) stores 
therein said weight values arrayed in the order in which said weight values are accessed. 

18. An information processing system according to claim 15. further including learning means (800, 
999) provided separately from said neural net, said learning means being so arranged as to be capable of 
being restored from faulty state. 

25 19. An information processing system according to claim 18, wherein said learning means is imple- 
mented by a program of a computer (550). 

20. An information processing system according to one of claims 1 to 19, wherein some of said blocks 
are formed in a semiconductor wafer (500). 

21. An information processing system according to claim 20, wherein a plurality of said semiconductor 
30 wafers are connected to one another. 

22. An information processing system, comprising: 

a plurality of neurons (100), wherein a weight value is provided for each of said neuron, said weight value 
being determined in relation to the other neurons, the output of each of said neurons being determined in 
accordance with a total sum of products of the associated weight value and the output values of the other 
35 neurons; 

an address bus (302) for transmitting an address signal for specifying sequentially one by one said plurality 
of the neurons; 

a data bus (300) for transmitting the output signal of each of said neuron; and 

means (107, 107 , 105. 106) for allowing each of said neurons to send out the output signal thereof only 
40 when it is specified by said address signal. 

23. An information processing system according to claim 22, wherein a state is provided in which none 
of said neurons is designated in the course of a round of said address signal. 

24. An information processing system according to claim 23, wherein each of said neurons includes a 
circuit (106) for storing temporarily the output of the associated neuron. 

45 25. An information processing system according to one of claims 22 to 24, wherein said plurality of the 
neurons are grouped into a plurality of groups (700. 701 -709), and wherein said address buses and said 
data buses are constituted, respectively, by first buses (301- 305) each connecting the neurons within each 
of said groups and second buses (311 - 315) for connecting said first buses. 

26. An information processing system according to claim 25, wherein said second bus is wired by using 
so a conductor having a greater width than that for said first bus. 

27. An information processing system according to claim 25, further including a clock bus (303) for 
supplying a clock signal to each of said neurons, wherein said transmission between said first bus and said 
second bus is performed in synchronism with said clock signal. 

28. An information processing system according to one of claims 22 to 27, wherein some of said 
55 neurons is formed on a semiconductor wafer (500). 

29. An information processing system according to one of claims 22 to 28, wherein each of said neuron 
includes address translation means (90) for performing a predetermined translation for the address signal 
inputted for thereby outputting onto said address bus an address signal differing from said inputted address 
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signal. 

30. An information processing system according to claim 29, wherein said address translation means is 
constituted by means (108) for adding or subtracting at least one bit to or from said inputted address signal. 

31. An information processing system according to claim 29. wherein each of said neurons includes an 
s address decoder (107) which responds to the address signal which is same as a predetermined signal. 

32. An information processing system according to one of claims 22 to 31 , wherein each of said 
neurons (100) includes memory means (101) for storing therein weight values in a number smaller than that 
of said neurons in the order starting from the maximum absolute value of said weight. 

33. An information processing system, comprising: 

w a plurality of semiconductor integrated circuits (100) of a same configuration provided on one and the same 
semiconductor substrate (500); 

a plurality of buses (311 - 315; 316) connected in common to said plurality of the semiconductor integrated 
circuits; 

wherein at least one (311 - 315) of said plurality of the buses connects intact those signals which make 
75 appearance at corresponding positions within said plurality of the semiconductor integrated circuits; and 
wherein at least other one (316) of said plural buses connects the signals at corresponding positions within 
said plural semiconductor integrated circuits by shifting said signals to one another. 

34. An information processing system according to claim 33, wherein said bus for connecting the 
signals by shifting them transmits the address signal for selecting a specific one from said plurality of the 

20 semiconductor integrated circuits. 

35. An information processing system according to claim 34, wherein each of said semiconductor 
integrated circuits includes a storage circuit (1 56) for storing temporarily the output of the associated 
semiconductor integrated circuit. 

36. An information processing system according to claim 35, wherein said plurality of the semiconduc- 
25 tor integrated circuits are provided on said semiconductor substrate, being grouped into a plurality of 

groups (700). and wherein said plurality of the buses are constituted by buses (311 - 315) of a first 
hierarchical level connected to the intra-group semiconductor circuits and buses (321 -325) of a second 
hierarchical level connected to said buses of said first hierarchical level. 

37. An information processing system according to claim 36, said plurality of buses including a bus 
30 (303) for transmitting a clock signal, and wherein signal transmission between said buses of the first and 

second hierarchical levels is performed in synchronism with said clock signal. 

38. An information processing system according to claim 36, wherein the bus (321 - 325) of said 
second hierarchical level is wired by using a conductor having a greater width than that of the bus of said 
first hierarchical level. 

as 39. An information processing system according to claim 33, wherein each of said semiconductor 
integrated circuits includes an address decoder (107) which responds to the same address signal as a 
predetermined signal. 

40. An information processing system according to claim 33, wherein each of said semiconductor 
integrated circuits* includes a memory (101) for storing weight values in a number smaller than that of said 

40 semiconductor integrated circuits sequentially in the order starting from the maximum absolute value of the 
weight. 

41. An information processing system of a structure in which semiconductor integrated circuits (100) are 
implemented on a wafer (500) by forming repetitively thereon at least a functional block (100), comprising: 
means (302) for discriminatively identifying said functional blocks from one another; 

45 means (306. 307) for inputting a first address signal to a first one of said functional blocks; and 

means (108) for adding or subtracting "1 " to or from at least one bit of said first address signal for thereby 
generating a second address signal differing from said first address signal, said second address signal 
being outputted to a second one of said functional blocks. 

42. An information processing system, comprising: 

so a wafer scale semiconductor integrated circuit of a structure in which at least one functional block (100) is 
repetitively employed on a wafer; 

means (302) for discriminatively identifying said functional blocks from one another; 

means (306. 307; 108) for inputting a first address signal to a first one of said functional blocks and adding 
or subtracting "1" to or from at least one bit of said first address signal for thereby generating a second 
55 address signal differing from said first address signal, said second address signal being outputted to a 
second one of said functional blocks; and 

an address decoder (107) incorporated in each of said functional blocks, said address decoder responding 
to the same address signal as a predetermined signal. 
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43. An information processing system, comprising: 

a wafer scale semiconductor integrated circuit in which at least one functional block is repetrtively employed 
on a semiconductor wafer; 

means for discriminatively identifying said functional blocks from one another; 
5 means for inputting a first address signal to a first one of said functional blocks and adding or subtractmg 
"1" to or from at least one bit of said first address signal for thereby generating a second address signal - 
differing from said first address signal, said second address signal being outputted to a second one of said 
functional blocks; 

a plurality of address signal lines (203) for transmitting by multiplexing said address signal; and 
ro means (11) incorporated in each of said functional blocks for performing a majority logic on the address 
signals inputted from said plurality of the address signal lines. 

44. An information processing system, comprising: 

a wafer scale semiconductor integrated circuit in which at least one functional block is repetitively employed 
on a semiconductor wafer; 

75 means for discriminatively identifying said functional blocks from one another; 

means for inputting a first address signal to a first one of said functional blocks and adding or subtracting 
"1" to or from at least one bit of said first address signal for thereby generating a second address signal 
differing from said first address signal, said second address signal being outputted to a second one of said 
functional blocks; . 

20 a data bus (300) for transmitting the output values of said functional blocks; and 

control means (12) included in each of said functional blocks for comparing the address s.gnal inputted to 
the associated functional block with a predetermined signal for thereby allowing said associated functional 
block to send out its own output value onto said data bus when coincidence occurs between said address 
signal and said predetermined signal. 

25 45. An information processing system, comprising: 

a wafer scale semiconductor integrated circuit (500) of such a structure in which at least one functional 
block (100) is repetitively employed, wherein at least one of said functional blocks includes means (220) for 
calculating an neuron model to be outputted. said functional blocks being mutually interconnected to form a 

30 rn^nsfoMnputting a first address signal to a first one of said functional blocks and adding or subtracting 
"1" to or from at least one bit of said first address signal for thereby generating a second address s.gnal 
differing from said first address signal, said second address signal being outputted to a second one of said 

functional blocks; . 
a plurality of signal lines for transmitting by multiplexing said address signals; and 
35 means (1 1) included in each of said functional means for performing a majority logic on the address signals 
inputted from said signal lines. 

46. An information processing system, comprising: 

a wafer scale semiconductor integrated circuit (500) in which at least one functional block (100) is 

repetitively employed on a semiconductor wafer; 
40 means for discriminatively identifying said functional blocks from one another; and 

means for inputting a first address signal to a first one of said functional blocks and adding or subtractmg 
"1" to or from at least one bit of said first address signal for thereby generating a second address signal 
differing from said first address signal, said second address signal being outputted to a second one of said 

functional blocks; . ^h,*^* 

45 wherein said functional blocks are connected in a matrix array by wiring conductors for said address 

signals. 

47. An information processing system, comprising: 

a plurality of neurons (100); . 

wherein weight values determined individually are prepared for each of the neurons in relation to the other 

ito^'SSi of said neurons, produces an output value determined in accordance with a total sum of 
products of the associated weight value and the output values of said other neurons, said wetght values 
being stored in a semiconductor storage device incorporated in the neuron destined to receive the s.gnal. 

48. An information processing system, comprising: 
55 a plurality of neurons (100); 

wherein weight values determined individually are prepared for each of the neurons in relation to the other 

IlSSn'^h.of said neurons produces an output value determined in accordance with a total sum of 
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products of the associated weight value and the output values of said other neurons, said weight values are 
stored in a predetermined number in the order starting from the greatest absolute value on the side for 

signal reception. . 

49. An information processing system, according to claim 47 or 48, wherein said weight values are 
stored as arrayed in the order in which said weight values are accessed. 

50. An information processing system, comprising: 

a plurality of blocks (100) each including a first circuit (101) having a storage function, a second circuit (102) 
having a multiplication function and a third circuit (103) having an addition function; 

wherein in each of said blocks, said second circuit and/or said third circuit are provided in common to a 
plurality of the other blocks, said second circuit determines products of a value stored in said first circuit 
and outputs of the other blocks, and said third circuit determines a total sum of said products, said total 
sum representing the output value of the block. . 

51. An information processing system, including a learning circuit (800; 999) of neurons is provided 
separately from a neural network in such a structure as to be capable of restoring from a faulty state. 

52. An information processing system according to claim 51. wherein said learning circuit is provided in 
a number not greater than that of the neurons. 

53. A semiconductor integrated circuit device, comprising: 

a first circuit (100;;=700) capable of performing operation even when fault should present in the circuit; and 
a second circuit (800. 999) incapable of performing operation when a fault exists; 
wherein means (550) for remedying the fault is provided only for said second circuit. 

54. A semiconductor integrated circuit device, comprising: 

a plurality of neural circuits (100) each having no learning circuit and formed on a semiconductor wafer; 
wherein a learning circuit (800, 999) to be used in common by said plurality of neural circuits and having a 
structure capable of remedying a fault is formed simultaneously with said plurality of neural circuits. 

55. A neural network system, comprising: 

at least a semiconductor integrated circuit device (500) having a plurality of neural circuits (100) each 
including no learning circuit; and 

a computer (550) electrically connected to said semiconductor integrated circuit device; 
wherein learning of said neural circuits is performed by a program running on said computer. 
, 56. A neural network system according to claim 55, wherein a plurality of said semiconductor wafers are 
interconnected. 

57. A neural network system, comprising: 

a plurality of neuron models- interconnected at multiple stages; 

each of said neuron models including: ♦-♦^ miM i 
s at least one synapse model (11) having at least one first input terminal, at least one first output terminal, 
means for holding weight values corresponding to said first input terminals, respectively, and means for 
arithmetically determining products of the input values applied to said first input terminals, respectively, and 
said weight values corresponding to said input values, respectively, said products being outputted to said 
first output terminals; and 

40 a cell body model (1b; 1c; 10) having at least one second input terminal, at least one second output 
terminal and means for receiving at said second input terminals said products each determ.ned arithmet- 
ically for each of said first input terminals in said synapse model, adding said products and performing a 
non-linear or linear function transformation on the sum value resulting from said addit.on by reading a 
memory having output values written at addresses corresponding to the sum values, wherein the results of 

45 said transformation being outputted from said second output terminal. 

58. A neural network system, comprising: 

a plurality of neuron models (100) connected to one another; 

each of said neuron models includes; . . i 

at least one first output terminal, means for holding weight values corresponding to said first input terminals, 
respectively. and>means for arithmetically determining products of the input values applied to said first input 
terminals, respectively, and said weight values corresponding to said input values, respectively. sa.d 
products being outputted to said first output terminals; and 

a cell body model (lb; 1c; 10) having at least one second input terminal, at least one second output 
terminal and means for receiving at said second input terminals said products each determ.ned arithmet- 
ically for each of said first input terminals, adding said products and performing a non-linear or linear 
function transformation on the sum value resulting from said addition, the result of said transformation being 
expressed at least in three values to be outputted to said second output terminals; 

wherein data transmission among said neuron models being performed by using a digital signal, while 
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arithmetic operation for determining said product and said sum in said neuron mode, is performed by using 
an analogue signal. 
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(neurons) receive the information on the data bus as 
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given functional blocks (neurons) to other given func- 
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